Information processing device and information processing method

ABSTRACT

The information processing device includes a factor determination unit configured to determine whether the behavior of an assisted person is an abnormal behavior of a dementia factor based on (1) information on a dementia level of the assisted person and (2) at least one of an environmental information, an excretion information, and a sleep information of the assisted person, and a support information output unit configured to output the support information to support an assistance of the assisted person by a caregiver based on the determination result of the factor determination unit and sensor information that is a sensing result about the assisted person or the caregiver assisting the assisted person.

TECHNICAL FIELD

The present invention relates to an information processing device, aninformation processing method, etc. This patent application claims thebenefit of priority to Japan Patent Application Serial No. 2021-032143,filed Mar. 1, 2021, which is incorporated by reference herein in itsentirety.

BACKGROUND ART

Traditionally, systems used in medical settings and nursing homes areknown. Patent Document 1, for example, discloses a technique forinstructing a method of assistance to move the assisted person.

CITATION LIST Patent Literature

-   Patent Document 1: Japanese Patent Laid-Open No. 2007-233471

SUMMARY OF THE INVENTION Technical Problem

The information processing device and the information processing methodthat appropriately support the assistance of the assisted person by thecaregivers

Solution to a Problem

The information processing device according to the present embodimentincludes a factor determination unit configured to determine whether thebehavior of an assisted person is an abnormal behavior of a dementiafactor based on (1) information on the dementia level of the assistedperson and (2) at least one of the environmental information, theexcretion information, and the sleep information of the assisted person,and a support information output unit configured to output the supportinformation to support an assistance of the assisted person by acaregiver based on the determination result of the factor determinationunit and sensor information that is a sensing result about the assistedperson or the caregiver assisting the assisted person.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows an example of the configuration of an informationprocessing system including an information processing device.

FIG. 2A shows an example of a nursing bed, which is a care device.

FIG. 2B shows an example of a lift, which is a care device.

FIG. 2C shows an example of a sensing device.

FIG. 2D shows an example of a sensing device.

FIG. 3 shows an example of a server system configuration.

FIG. 4 shows an example of the configuration of a mobile terminaldevice.

FIG. 5 is an illustration of the neural network.

FIG. 6 is an example of an input and output of the neural network for afactor determination.

FIG. 7 is a flow chart explaining a learning process for determiningfactors.

FIG. 8 is a flow chart explaining a factor determination process.

FIG. 9 shows an example of an input and output of the neural network fora support information output.

FIG. 10 shows a configuration example of a neural network for outputtinga support information.

FIG. 11 shows a configuration example of a neural network for outputtinga support information.

FIG. 12 shows a configuration example of a neural network for outputtinga support information.

FIG. 13 shows an example of the relationship between a neural networkfor determining factors and a neural network for outputting a supportinformation.

FIG. 14 shows an example of a first association information.

FIG. 15 shows an example of a second association information.

FIG. 16 shows an example of a third association information.

FIG. 17 shows an example of the settings screen.

FIG. 18 is a flowchart illustrating a setting process.

FIG. 19 is a flowchart describing the processing for outputting eachsupport information.

FIG. 20 is a flow chart illustrating a start decision of a assistancesequence.

FIG. 21 is a flow chart illustrating a meal assistance sequence.

FIG. 22 is a flow chart illustrating a excretion assistance sequence.

FIG. 23 is a flow chart illustrating a transfer assistance sequence.

FIG. 24A is a diagram illustrating the transitions between multipleassistance sequences.

FIG. 24B is a diagram illustrating the transitions between multipleassistance sequences.

FIG. 25A shows an example of a settings screen.

FIG. 25B shows an example of a display screen for adding data.

FIG. 25C shows an example of a display screen for determining an inputdata.

FIG. 25D shows an example of a display screen for presenting learningresults.

FIG. 26 shows a basic configuration example of a neural network in thisembodiment.

FIG. 27 illustrates a determining process to determine a structure of aneural network by classification.

FIG. 28 is a detailed example of input data in supporting the assistanceof a assisted person by a caregiver.

FIG. 29 is a detailed example of input data in supporting the assistanceof a assisted person by a caregiver.

FIG. 30 is a detailed example of input data in supporting the assistanceof a assisted person by a caregiver.

FIG. 31 is a detailed example of input data in supporting the assistanceof a assisted person by a caregiver.

FIG. 32 is a detailed example of input data in supporting the assistanceof a assisted person by a caregiver.

FIG. 33 is a detailed example of input data in supporting the assistanceof a assisted person by a caregiver.

FIG. 34 is a detailed example of input data in supporting the assistanceof a assisted person by a caregiver.

FIG. 35 is a detailed example of input data in supporting the assistanceof a assisted person by a caregiver.

FIG. 36 is a detailed example of input data in supporting the assistanceof a assisted person by a caregiver.

FIG. 37 is a detailed example of input data in supporting the assistanceof a assisted person by a caregiver.

FIG. 38 is a detailed example of input data in supporting the assistanceof a assisted person by a caregiver.

FIG. 39 is a detailed example of input data in supporting the assistanceof a assisted person by a caregiver.

FIG. 40 is a detailed example of input data in supporting the assistanceof a assisted person by a caregiver.

FIG. 41 is a detailed example of input data in supporting the assistanceof a assisted person by a caregiver.

FIG. 42 is a detailed example of input data in supporting the assistanceof a assisted person by a caregiver.

FIG. 43 is a detailed example of output data in support of a mealassistance.

FIG. 44 is a detailed example of output data in support of an excretionassistance.

FIG. 45 is a detailed example of output data in support of a transferand mobility assistance.

DESCRIPTION OF EMBODIMENT

Hereafter, the present embodiment will be described with reference tothe drawings. In the case of drawings, identical or equivalent elementsshall be denoted by the same symbol, and duplicate descriptions shall beomitted. It should be noted that this embodiment described below doesnot unreasonably limit the contents of the claims. Also, not all of theconfigurations described in the present embodiment are mandatoryconfiguration requirements.

1. System Configuration Example

FIG. 1 is a configuration example of an information processing system 10including an information processing device according to this embodiment.The information processing system 10 according to this embodimentprovides instructions to the caregivers so that appropriate assistancecan be provided regardless of the skill level of the caregivers bydigitizing the “intuition” or “tacit knowledge” of the caregivers, forexample, in a care facility. The information processing system 10 shownin FIG. 1 includes a server system 100, a caregiver device 200, a caredevice 300 which is used for a care, and a sensor group 400. However,the configuration of the information processing system 10 is not limitedto FIG. 1 , and various modifications such as omitting a part or addingother configurations are possible. In addition, the fact thatmodifications such as omission or addition of a configuration can becarried out is the same in FIGS. 3 and 4 , which will be describedlater.

The information processing device of this embodiment corresponds to, forexample, the server system 100. However, the method of this embodimentis not limited to this, and the processing of the information processingdevice may be executed by distributed processing using the server system100 and other devices. For example, the information processing device ofthis embodiment may include the server system 100 and the caregiverdevice 200. An example in which the information processing device is theserver system 100 is described below.

The server system 100 is connected to the caregiver device 200, a caredevice 300, and a sensor group 400 for example via the network NW. Thenetwork NW here is, for example, a public communication network such asthe Internet, but may also be a LAN (Local Area Network). For example,the caregiver device 200, a care device 300 and a sensor group 400 areplaced in a nursing home, etc. The server system 100 performs processingbased on the information from the sensor group 400, outputs informationto the caregiver device 200 based on the processing results, andremotely controls the care device 300, etc based on the processingresults.

In the FIG. 1 , each of the caregiver device 200, the care device 300,and the sensor group 400 can communicate with the server system 100through the network NW, but this is not limited. For example, a relaydevice (not shown) may be provided in a nursing home. The relay deviceis a device capable of communicating with the server system 100 throughthe network NW. The information output by the sensor group 400 isaggregated by a relay device using a LAN in the nursing home, and therelay device may transmit the information to the server system 100.Information from the server system 100 is transmitted to the relaydevice, and the relay device may transmit necessary information to thecaregiver device 200 or the care device 300. For example, in nursinghomes, it is assumed that multiple caregiver devices 200 and multiplecare devices 300 will be used simultaneously. The relay device mayperform processing to select the caregiver device 200 or the care device300 to which the information from the server system 100 is to betransmitted. Alternatively, the relay device may be a manager terminalused by the manager of the nursing home and may operate based on theoperator's input. For example, the information from the server system100 is displayed on the display of the relay device, and the manager whosees the displayed result may select the caregiver device 200 or caredevice 300 as a destination device. In addition, as described above,various modifications can be made to the information processing deviceof this embodiment, and for example, the above relay device may beincluded in the information processing device.

The server system 100 may be a single server or may include multipleservers. For example, the server system 100 may include a data baseserver and an application server. The database server stores variousdata to be described later using FIG. 3 . The application serverperforms the processing described later using FIG. 7 , FIG. 8 , FIG. 18to FIG. 23 , etc. The multiple servers here may be physical servers orvirtual servers. If a virtual server is used, the virtual server may belocated on one physical server or distributed among multiple physicalservers. As described above, the detailed configuration of the serversystem 100 in this embodiment can be modified in various ways.

The caregiver device 200 is a device used by a caregiver who assists anassisted person (Patients, residents) in a nursing home, etc., topresent information to the caregiver or to input information by thecaregiver. For example, the caregiver device 200 may be a device carriedor worn by the caregiver. For example, the caregiver device 200 includesa mobile terminal device 210 and a wearable device 220. The mobileterminal device 210 is, for example, a smartphone, but may be any othermobile device. The wearable device 220 is a device that can be worn bythe caregivers, for example, an earphone or headphone and a headsetcontaining a microphone. The wearable device 220 may be a glasses-typedevice, a wristwatch-type device, or a device of another shape. Thecaregiver device 200 may be another device such as a PC (PersonalComputer).

A care device 300 is a device used to provide care (includingassistance) for an assisted person in a nursing home, etc. Whereas thecaregiver device 200 is primarily a device for presenting information tothe caregiver, the care device 300 is a device for directly assistingthe assisted person. For example, the care device 300 may include anursing bed 310 which can change an angle of bottoms (which may beplate-shaped or mesh-shaped, regardless of shape) and a height, and alift 320 for transferring the assisted person from the nursing bed 310to a wheelchair, etc. The care device 300 may also include otherequipment such as a wheelchair, a walker, rehabilitation equipment, anda serving cart to serve meals.

FIG. 2A shows an example of a nursing bed 310. The nursing bed 310 iscapable of changing the height and angle of the multiple bottoms,respectively. This makes it possible to flexibly change the posture ofthe assisted person lying on the nursing bed 310. FIG. 2B is an exampleof the lift 320. The lift 320 is a device used, for example, to transfera care recipient who has a low ADL (Activity of Daily Living) ratingindex and is difficult to transfer by hand.

The sensor group 400 includes a plurality of sensors located in anursing home, etc. The sensor group 400 may include a motion sensor 410,an imaging sensor 420, and an odor sensor 430. The motion sensor 410 maybe an acceleration sensor, a gyro sensor or any other sensor capable ofdetecting motion. The motion sensor 410 may be a sensor that detects themotion of the assisted person or a sensor that detects the motion of thecaregiver. The imaging sensor 420 is a sensor that converts an objectimage formed through a lens into an electrical signal. The odor sensor430 is a sensor that detects and quantifies odor. The sensor group 400can also include various sensors such as temperature sensors, humiditysensors, illuminance sensors, magnetic sensors, position sensors,barometric pressure sensors, etc.

FIG. 1 shows the caregiver device 200, the care device 300, and thesensor group 400 separately. For example, the sensors included in thesensor group 400 may be located in living rooms, dining rooms, hallways,stairs, etc., in nursing homes. For example, a camera including theimaging sensor 420 is placed at each location in a nursing home. Sensingdevices may also be used to sense information needed for caregiving. Notonly the necessary information is sensed and but also the locationinformation is detected by providing the sensors at each location in anursing home.

For example, FIG. 2C shows a example of the sensing device 440 placed onthe mattress of a nursing bed 310. The sensing device 440 shown in FIG.2C includes, for example, the odor sensor 430 to detect whether theassisted person has excreted. The sensing device 440 may be capable ofdetermining whether the assisted person is ill from body odor or breath.FIG. 2D also shows an example of a sensing device 450 placed under amattress (placed between the nursing bed 310 and the mattress) on thenursing bed 310. The sensing device 450 shown in FIG. 2D includes, forexample, a pressure sensor and can detect the heart rate, respiratoryrate and activity of the assisted person. The sensing device 450 may beable to determine whether the assisted person is in a sleep state or notand whether the assisted person is in a nursing bed.

However, the method of this embodiment is not limited to the aboveexamples, and the sensors included in the sensor group 400 may beprovided in the caregiver device 200 or the care device 300. Forexample, as the sensors included in the sensor group 400, cameras,accelerometers, gyro sensors, GPS (Global Positioning System) sensors,etc. in the mobile terminal device 210 may be used. In addition, thecare device 300 may be provided with a motion sensor for detecting theposture of the care device 300, and a camera for imaging the assistedperson or the caregivers using the care device 300, etc.

FIG. 3 is a block diagram showing a detailed configuration example ofthe server system 100. The server system 100 includes, for example, aprocessing unit 110, a storage unit 120, and a communication unit 130.

The processing unit 110 of this embodiment includes the followinghardware. The hardware may include at least one of a circuit forprocessing digital signals and a circuit for processing analog signals.For example, the hardware may be one or more circuit devices mounted ona circuit board or one or more circuit elements. One or more circuitdevices are, for example, IC (Integrated Circuit) or FPGA(field-programmable gate array). One or more circuit elements are, forexample, resistors, capacitors, etc.

The processing unit 110 may be realized by the following processors. Theserver system 100 of this embodiment includes a memory for storinginformation and a processor operating on the information stored in thememory. The information includes, for example, programs and variouskinds of data. The processor includes the hardware. The processor canuse a variety of processors such as a CPU (Central Processing Unit), GPU(Graphics Processing Unit), and DSP (Digital Signal Processor). Thememory may be a semiconductor memory such as SRAM (Static Random AccessMemory), DRAM (Dynamic Random Access Memory), or flash memory, or aregister, a magnetic storage device such as a Hard Disk Drive (HDD), oran optical storage device such as an optical disk device. For example,the memory stores instructions that can be read by a computer, and ifthe processor executes the instructions, the functions of the processingunit 110 may work. The instructions described above may be theinstruction set that makes up the program, or the instruction thatinstruct the processor's hardware circuitry to operate.

The processing unit 110 includes a factor determination unit 111, asupport information output unit 112, a setting unit 113, and a learningunit 114.

The factor determination unit 111 determines whether the assistedperson's behavior is an abnormal behavior of the dementia factor onaccordance with an input including at least dementia level informationof the assisted person. For example, the factor determination unit 111determines whether the behavior of the assisted person is an abnormalbehavior of the dementia factor based on (1) the dementia levelinformation of the assisted person and (2) at least one of theenvironmental information of assisted person, the excretion informationof the assisted person and the sleep information of the assisted person.The details of each information will be described later.

The support information output unit 112 outputs support information tosupport the assistance of the assisted person by the caregiver based onthe determination result output by the factor determination unit 111 andthe sensor information which is the sensing result about the assistedperson or the caregiver who assists the assisted person. The detail ofthe support information is provided below.

The setting unit 113 performs setting processing when using theinformation processing system 10 according to this embodiment. Forexample, a caregiver who is a user of the information processing system10 may be able to set which information should be output from a largenumber of pieces of support information. In this case, as will bedescribed later with reference to FIG. 17 and FIG. 18 , the setting unit113 performs processing such as accepting the setting operation by thecaregiver and updating the setting information. The setting unit 113 mayalso perform setting processing to add user-specific custom supportinformation as the output. Detailed examples will be described laterusing FIGS. 25A to 25 D, etc.

The learning unit 114 outputs the learned model by performing a machinelearning based on the training data. The machine learning describedabove is, for example, supervised learning. The training data insupervised learning is a data set made an association between the inputdata corresponding to the input of the model and the correct answer datarepresenting the appropriate output data when the input data is input.The learning unit 114 may generate a learned model, for example, byperforming the machine learning using a neural network. Hereafter, theneural networks are referred to as NN. For example, the learning unit114 performs processing to generate a factor determination NN 121 and asupport information output NN 122. Details of the processing in thelearning unit 114 will be described later. However, the machine learningis not always required in this embodiment, and the learning unit 114 canbe omitted. In the case of the machine learning is performed, thelearning process can be executed in a learning device different from theserver system 100, and the learning unit 114 can be omitted in this caseas well.

The storage unit 120 is a work area of the processing unit 110 andstores various information. The storage unit 120 can be realized by avariety of memories, and the memory may be a semiconductor memory suchas SRAM, DRAM, ROM, flash memory, etc, a register, a magnetic storagedevice, or an optical storage device.

The storage unit 120 stores information used for processing in thefactor determination unit 111 and information used for processing in thesupport information output unit 112. For example, the storage unit 120may store the factor determination NN 121 acquired by the machinelearning using NN and a support information output NN 122 acquired bythe machine learning using NN. Here, the factor determination NN 121 andthe support information output NN 122 include the parameters used forthe operation using the structure in addition to the informationspecifying the structure of the NN. A parameter is specifically a weightwhose value is determined by the machine learning.

The storage unit 120 may also store a first association information 123,a second association information 124, and a third associationinformation 125. The first association information 123 is informationthat associates a caregiver with information indicating whether or notto output each support information to the caregiver. The secondassociation information 124 is information that associates the supportinformation with the sensor information required to output the supportinformation. The third association information 125 is information thatassociates a given nursing home with sensor information that can beacquired in the nursing home. The Detailed examples of each associationinformation will be described later with reference to FIGS. 14 to 16 .The storage unit 120 may store other information.

The communication unit 130 is an interface for communication via anetwork NW and includes, for example, an antenna, a radio frequency (RF)circuit, and a baseband circuit. The communication unit 130 may operateaccording to control by the processing unit 110 or may include aprocessor for communication control different from the processing unit110. The communication unit 130 is an interface for performingcommunication according to, for example, TCP/IP (Transmission ControlProtocol/Internet Protocol). However, various modifications can be madeto the detailed communication system.

FIG. 4 is an example of the caregiver device 200 and a block diagramshowing a detailed configuration example of the mobile terminal device210. The mobile terminal device 210 includes, for example, a processingunit 211, a storage unit 212, a communication unit 213, a display unit214, and an operation unit 215.

The processing unit 211 is composed of a hardware including at least oneof a circuit for processing digital signals and a circuit for processinganalog signals. The processing unit 211 may also be realized by aprocessor. It is possible to use a variety of processors such as CPU,GPU, and DSP. The processor executes instructions stored in the memoryof the mobile terminal device 210, thereby realizing the function of theprocessing unit 211 as processing.

The storage unit 212 is a work area of the processing unit 211 and isrealized by various memories such as SRAM, DRAM, ROM, etc.

The communication unit 213 is an interface for communication via anetwork NW and includes, for example, an antenna, an RF circuit, and abaseband circuit. The communication unit 213 communicates with theserver system 100 through, for example, the network NW.

The display unit 214 is an interface for displaying various kinds ofinformation and may be a liquid crystal display, an organic EL display,or an another type display. The operation unit 215 is an interface thataccepts user operations. The operating unit 215 may be a button or thelike provided in the mobile terminal device 210. The display unit 214and the operation unit 215 may be a touch panel constructed as one unit.

Also, the mobile terminal device 210 may have a light emitting part, avibration part, a sound output part, or other part which is not shown inFIG. 4 . The light-emitting part is, for example, LED (light emittingdiode), which emits light. The vibrating part is, for example, a motor,which gives an alarm by vibration. The sound output unit is a speaker,for example, and provides sound notification. Also, as described above,the mobile terminal device 210 may include sensors included in thesensor group 400.

2. Factor Determination and Support Information Output

The information processing device of this embodiment performs aprocessing to determine the factors of the behavior of the assistedperson, and a processing to output support information supporting theassistance of the assisted person by the caregiver. In this way, it ispossible to have the caregiver provide appropriate assistance accordingto the assisted person by taking into account factors such as dementia.The machine learning is described below as a detailed example of amethod for determining the factors and performing the supportinformation output processing. However, the method of this embodiment isnot limited to the one using the machine learning, and variousmodifications can be performed. In the following, we describe an exampleof using NN as the machine learning, but other methods such as supportvector machines (SVMs) may be used for the machine learning, or othermethods developed from NN or SVM may be used.

2.1 Brief Description of the NN

FIG. 5 shows an example of the basic structure of the NN. One circle inFIG. 5 is called a node or neuron. In the example of FIG. 5 , the NN hasan input layer, two or more intermediate layers, and an output layer.The input layer is I, the intermediate layers are H1 and Hn, and theoutput layer is O. In the example of FIG. 5 , the number of nodes in theinput layer is 2, the number of nodes in the middle layer is 5, and thenumber of nodes in the output layer is 1. However, the number of layersin the middle layer and the number of nodes contained in each layer canbe modified variously. FIG. 5 also shows an example in which each nodeincluded in a given layer is connected to all nodes included in the nextlayer, and various modifications can be made to this configuration aswell.

The input layer accepts the input value and outputs value to theintermediate layer H1. In the example of FIG. 5 , the input layer Iaccepts two kinds of input values. Each node in the input layer mayperform some processing for the input value and output the value afterthe processing.

In the NN, a weight is set between two connected nodes. W1 in FIG. 5 isthe weight between the input layer I and the first intermediate layerH1. W1 represents the set of weights between a given node in the inputlayer and a given node in the first intermediate layer. For example, W1in FIG. 5 is information containing 10 weights.

Each node of the first intermediate layer H1 preforms an operation toweighted add the output of the node of the input layer I connected tothe node using the weight W1, and further operation to add the bias. Inaddition, at each node, the output of the node is determined by applyingthe activation function, which is a nonlinear function, with thesummation result. The activation function may be a ReLU function, asigmoid function, or any other function.

The operation is same for subsequent layers. That is, in a given layer,the output of the preceding layer is weighted and added with the weightW, and the bias is added and then the output to the next layer iscalculated by applying an activation function. The NN treats the outputof the output layer as the output of the NN.

As we can see from the above description, to obtain the desired outputdata from the input data using NN, it is necessary to set theappropriate weights and biases. In learning, we should prepare thetraining data made an association between a given input data and thecorrect data representing the correct output data in the input data. Thelearning process of the NN indicates a process for finding the mostprobable weight based on the training data. In the learning process ofthe NN, various learning methods such as backpropagation are known. Inthe present embodiment, since these learning methods can be widelyapplied, a detailed description is omitted.

Also, the NN is not limited to the configuration shown in FIG. 5 . Forexample, as the NN, a convolutional neural network (CNN: convolutionalneural network) may be used. The CNN has a convolution layer and apooling layer. The convolution layer performs convolution operations.Convolution operations described here are specifically filtering. Thepooling layer performs processing to reduce the vertical and horizontalsizes of the data. In the CNN, the characteristics of the filter used inthe convolution operation are learned by performing learning processingusing an backpropagation method or the like. That is, the weights in theNN include the filter characteristics in the CNN. As the NN, a networkwith other configuration such as RNN (Recurrent neural network) may beused.

2.2 Factor Judgment

FIG. 6 illustrates the input and output data of the factor determinationNN 121 used for factor determination. The input data in factordetermination includes, for example, dementia level information. Theinput data also includes at least one of environmental information,sleep information and excretion information. FIG. 6 shows an examplewhere the input data includes all of the environmental information, thesleep information and the excretion information. The input data may alsocontain other information. For example, as shown in FIG. 6 , the inputdata may include medication information, dietary water information. Theconfiguration of the factor determination NN 121 is not limited to FIG.6 , and various modifications can be performed.

Dementia level information is information that represents the degree ofprogression of dementia in the assisted person. For example, thedementia level information may be a score on the Mini-Mental StateExamination (MMSE), a score on the revised Hasegawa's Brief IntelligenceScale (HDS-R), or other information representing the results of adementia test. The dementia level information may be based on brainimages obtained using computed tomography (CT) or magnetic resonanceimaging (MRI). For example, the dementia level information may be theresult of a doctor's diagnosis based on a brain image, the brain imageitself, or the result of some kind of image processing on the brainimage.

Environmental information is information that represents the livingenvironment of the assisted person. The environmental informationincludes temperature information representing the temperature of theliving environment of the assisted person, humidity informationrepresenting humidity, illuminance information representing illuminance,and barometric information representing barometric pressure. Forexample, a temperature sensor, a humidity sensor, an illuminance sensor,and a barometric pressure sensor are placed in the patient's living roomor a place regularly used, such as a dining room, and temperature,humidity, illuminance, and barometric pressure information are acquiredbased on the output of each sensor.

The environmental information may also include information related tosound. For example, a microphone is placed in a living environment suchas a living room, and the information collected by the microphone isused as environmental information. The environmental information can beinformation related to sound pressure or information representing theresults of frequency analysis. The environmental information may alsoinclude information about the time etc, when a particular sound occurs.

The environmental information may also include information related tothe nursing bed 310 used by the assisted person. The information aboutthe nursing bed 310 may refer to the model of the nursing bed 310, maybe specific information or information such as the type and firmness ofmattresses used in conjunction with the nursing bed 310. The informationabout the nursing bed 310 may also include information representing thedriving result of the nursing bed 310. For example, information such asthe angle and height of the bottom of the nursing bed 310 and the timewhen the nursing bed 310 drives may be used as the environmentalinformation.

The sleep information is information representing the sleep state of theassisted person. For example, sleep information may be detected using asensing device 450 or the like which is shown in FIG. 2D. The sleepinformation may also be detected using a wristwatch-type deviceincluding a photoelectric sensor or the like for detecting pulse rate.The sleep information includes, for example, information such as sleepstart time, wake-up time, daily sleep duration, sleep depth, number andtime of arousal during sleep, heart rate, respiratory rate and amount ofactivity during sleep.

The excretion information includes information representing the state ofexcretion of the assisted person. For example, the excretion informationmay be detected using the sensing device 440 which is shown in FIG. 2C.The sensing device 440 outputs the presence or absence of excretion ofthe assisted person, the type of excretion, and the timing when theexcretion is determined based on the odor sensor 430 for example. Theexcretion information includes information such as, for example, thenumber of excretions in a given period, the interval of excretion, andthe type of excretion. The excretion information may also includeinformation such as an imaged image of the diaper after excretion andcomments added by the caregiver.

The medication information is information that identifies the medicineadministered to the assisted person. For example, the medicationinformation is information that represents the name of the medicinetaken by the assisted person, dose, time of dose, etc. The medicationinformation may also include information on prescriptions issued to theassisted person.

Dietary water information is information that represents the food andwater taken by the assisted person. The dietary water information, forexample, includes the time of day you ate, the menu, and how much youactually ate. The dietary water information may also include informationidentifying ease of eating, such as firmness and size of ingredients.The dietary water information also includes the time of taking water,type of water (water, tea, etc.), and amount of taking water.

In the learning stage, the training data for creating the factordetermination NN 121 is acquired by associating the above input data ina prescribed period with the correct answer data. The prescribed perioddescribed above may be a fixed period such as one day. Alternatively,the prescribed period may be a period that is set on the basis of thetime of occurrence of any abnormal behavior if the assisted personbehaves abnormally.

The correct answer data may also be given by an expert with specializedknowledge, such as a physician and a doctor. If the assisted personbehaves abnormally, the expert makes a diagnosis of the assisted personand identifies the factors contributing to the abnormal behavior. Thecorrect answer data described here is information representing theidentified factors. For example, the correct answer data indicateswhether the behavior is triggered by a dementia factor, an environmentalfactor, a sleep disorder factor, or a excretion disorder factor. Forexample, in the case that one data set is defined as the result ofassociating the correct answer data with the input data corresponding toone period of one assisted person, the training data including a largenumber of data sets is acquired by increasing the number of assistedpersons and the target period.

The learning unit 114 of the server system 100 acquires the trainingdata for determining factors. Then, by performing the machine learningbased on the training data, the factor determination NN 121 isgenerated.

FIG. 7 is a flowchart illustrating the learning process for generatingthe factor determination NN 121. When this processing is started,firstly in a step S101, the learning unit 114 acquires the input datafor learning. The input data described here are same as described aboveand include, for example, the dementia level information, theenvironmental information, the sleep information and the excretioninformation. The input data may also include other information such asthe medication information and the dietary water information.

Also, in a step S102, the learning unit 114 obtain the correct answerdata which associates with the input data. For example, the learningunit 114 performs the processing of the steps S101 and S102 by readingout any one of the training data sets acquired in the learning stage.

In a step S103, the learning unit 114 performs processing to update theweight of the NN. Specifically, the learning unit 114 inputs the inputdata acquired in the step S101 into the factor determination NN 121, andacquires the output data by performing a forward operation using theweights at that stage. The learning unit 114 obtains an objectivefunction based on the output data and the correct answer data. Theobjective function described here is, for example, an error functionbased on the difference between the output data and the correct answerdata, or a cross-entropy function based on the distribution of theoutput data and the distribution of the correct answer data.

For example, if the output layer of the factor determination NN 121 is aknown softmax layer, the output of the output layer is probability datawhose sum is 1. For example, the output layer includes four nodes fromthe first node to the fourth node. The output value of the first noderepresents a “certainty that the behavior of the assisted person istriggered by a factor in dementia.” The output value of the second noderepresents a “certainty that the behavior of the assisted person istriggered by an environmental factor.” The output value of the thirdnode represents a “certainty that the behavior of the assisted person istriggered by a sleep disturbance factor.” The output value of the fourthnode represents a “certainty that the behavior of the assisted person istriggered by a factor in the excretion disorder.” The correct answerdata are data in which the value of the factor that is the correctanswer is 1 and the other values are 0. For example, if an expertdetermines that the dementia factor is a factor, data that thepossibility for the dementia factor corresponds to 1 and the possibilityof other three factors corresponds to 0 are used as correct data.

The learning unit 114 updates the weights so that, for example, theerror function decreases. As a weight updating method, theback-propagation method and the like described above are known, andthese methods can be widely applied in this embodiment.

In the step S104, the learning unit 114 determines whether or not toterminate the learning process. For example, multiple datasets includedin training data may be separated into a learning data and a validationdata. The learning unit 114 may complete the learning process whenprocessing to update the weights using all the learning data isperformed, or may complete the learning process when the correct answerrate by the validation data exceeds a prescribed threshold.

If the learning process is not completed, the learning unit 114 returnsto the step S101 to continue the process. That is, the learning unit 114reads a new data set from the training data and performs processing toupdate the weights based on the new data set.

When the learning process is completed, the learning unit 114 stores thefactor determination NN 121 at that stage in the storage unit 120 as alearned model. Note that FIG. 7 is an example of learning processing,and the method of this embodiment is not limited to this. For example,in the machine learning, the methods such as a batch learning are widelyknown, and these methods can be widely applied in this embodiment.

FIG. 8 is a flowchart explaining the processing of the factordetermination unit 111 in an inference stage. When this processing isstarted, firstly, in a step S201, the factor determination unit 111determines whether or not the assisted person has engaged in abnormalbehavior that is suspected to be dementia. The factor determination unit111 may automatically determine whether the behavior of the assistedperson is abnormal behavior based on the sensor information about theassisted person, etc. For example, the sensor group 400 includes amotion sensor 410, an imaging sensor 420, a microphone, etc., and thefactor determination unit 111 determines the presence or absence ofabnormal behavior by detecting the movement or vocalization of theassisted person. Alternatively, the caregiver may observe the movementof the assisted person himself or herself and input the observationresults using the caregiver device 200 or the like. In this case, thefactor determination unit 111 performs the processing of step S201 basedon the input of the caregiver. When it is determined that the assistedperson does not exhibit any abnormal behavior, the factor determinationunit 111 terminates the processing without performing a step S202 orlater.

When it is determined that the assisted person has behaved abnormally,in the step S202, the factor determination unit 111 acquires the inputdata concerning the assisted person. For example, the storage unit 120acquires and stores the dementia level information about the assistedperson, the sensor information collected by the sensor group 400, andthe like via the communication unit 130. The dementia level informationmay be obtained, for example, at a nursing home, and transmitted fromthe equipment at the nursing home to the server system 100. The factordetermination unit 111 performs processing to read out, as the inputdata, the information on the dementia level, the environmentalinformation, the sleep information, the excretion information and thelike corresponding to a prescribed period, among the collected dataconcerning the target assisted person.

In a step S203, the factor determination unit 111 reads the factordetermination NN 121 from the storage unit 120. Then, the input dataacquired in the step S202 is input to the factor determination NN 121,and the output data is obtained by performing a forward operation. Theoutput data of the factor determination NN 121 are, for example, 4probability values representing the certainty of each factor asdescribed above. The factor determination unit 111 determines, forexample, the factor that maximizes the probability value as the factorof the abnormal behavior of the assisted person. For example, if a valuerepresenting the certainty of a dementia factor is greater than thecertainty of the other 3 factors, the factor determination unit 111determines that the abnormal behavior is triggered by a dementia factor.The output of the factor determination unit 111 is not limited to this,and may be the 4 probability values themselves or a value calculatedbased on them.

The factor determination unit 111 periodically executes the processingshown in FIG. 8 , for example. The frequency of processing is arbitrarybut may be, for example, once a day. In this way, the presence orabsence of abnormal behavior of the assisted person and the factors inthe event of abnormal behavior can be determined on a regular basis. Forexample, the factor determination unit 111 may execute the processingwhich is shown in FIG. 8 every morning and determine the assistancepolicy for the day based on the processing result. In addition, variousmodifications can be made to the processing of the factor determinationunit 111, such as executing the processing which is shown in FIG. 8without waiting for the next processing timing, when the abnormalbehavior is observed in the assisted person.

In addition, as a separate processing from the processing using thefactor determination NN 121, an example of determining whether thebehavior of the assisted person is abnormal behavior has been describedabove (see Step S201 in FIG. 8 ). However, a NN for making adetermination including whether the behavior is abnormal may begenerated.

For example, the factor determination NN 121 may have the sensorinformation or the like representing the behavior of the assisted personin addition to the input data shown in FIG. 6 . The factor determinationNN 121 may include a node for outputting the “certainty of noabnormality in the behavior of the assisted person.” in addition to thenode for outputting the certainty of the four factors shown in FIG. 6 .In the learning phase, the training data is generated by using data inthe absence of abnormal behavior in addition to data in the presence ofabnormal behavior. Specifically, the correct answer data associated withthe input data includes data representing “no abnormal behavior.” Inthis case, the factor determination unit 111 estimates the presence orabsence of abnormal behavior and factors in the event of abnormalbehavior by inputting the input data to the factor determination NN 121.

2.3 Assistance Support

2.3.1 Input and Output

FIG. 9 is a diagram illustrating the schematic input data of the supportinformation output NN 122 used to output support information. As shownin FIG. 9 , the input data may include the sensor information. Thesensor information includes information sensing the assisted person orinformation sensing the assisted person. The sensor information isoutput from sensors included in the sensor group 400, for example.

In addition, the sensor information may include information that sensesthe living environment of the assisted person. The sensor information inthis case corresponds to the environmental information described above,for example. For example, the sensor information may include outputs oftemperature sensors, humidity sensors, illuminance sensors, barometricpressure sensors, microphones, etc.

The input data also includes attribute data on the assisted person, aswell as physical assessment data that represents a physical assessment.The attribute data of the assisted person includes information such asage, sex, height, weight, medical history, medication history, etc. Thephysical assessment data includes information such as ADL assessment,rehabilitation history, fall risk and pressure ulcer risk.

The input data also includes attribute data on the caregivers and dataon the nursing homes. The attribute data on the caregivers includes thecaregiver's age, sex, height, weight, assistance experience, and heldqualifications. The data on the nursing homes includes information suchas nursing schedules, the number and usage of care device 300, thenumber of the assisted person, and statistical data on the level of careneeds at the nursing home.

FIGS. 28 to 42 illustrate details of data used as input in supportingthe assistance of the assisted person by the caregivers in thisembodiment, and in a narrow sense, they show examples of input data ofthe support information output NN 122. As shown in the FIGS. 28 to 42 ,the input data in this embodiment can utilize various kinds ofinformation. It is not mandatory that all the input data shown in theFIGS. 28 to 42 be acquired, even if some information is omitted. Theother information not shown in the FIGS. 28 to 42 may be added.

In addition, in the case that assistance of the assisted person by thecaregivers is divided into multiple assistance actions, the output dataof the support information output NN 122 is information for supportingthe performance of each assistance action. For example, the output dataof the support information output NN 122 is support information fordetermining the start timing of the assistance action, the movement andvocalization during the assistance action, and the type and amount ofobjects to be provided to the assisted person.

FIGS. 43 to 45 are diagrams illustrating details of data used insupporting the assistance of the assisted person by the caregiver inthis embodiment, and in a narrow sense, they are diagrams illustratingexamples of support information that is output data of the supportinformation output NN 122.

FIG. 43 is an example of the support information that is output in mealassistance where the caregivers assist the assisted person to eat themeal. For example, in the meal assistance, the caregiver grasps thecharacteristics of the assisted person and explains them to the assistedperson himself/herself in an easy-to-understand manner, therebyfacilitating to execute the meal assistance. For example, in the case ofan assisted person who is characterized by poor chewing ability, it ispossible to take ways to prevent aspiration if the caregiver is aware ofthis, and it is also useful to guide the assisted person by saying, “wehave softened the rice, so let's chew it well.” The output data ofNumber 1 in FIG. 43 is support information to “convey thecharacteristics of the user” to the caregiver, and it may be datarepresenting the characteristics of the assisted person itself, or itmay be information converted to make it easy for the caregiver tounderstand. Also, as mentioned above, the caregiver may communicate thecharacteristics of the assisted person to the assisted person himself orherself, and the output data of Number 1 in the FIG. 43 may contain datafor that, also Number 2 and thereafter may be same as Number 1, and theoutput data shown in the FIG. 43 includes information to support thecaregiver's various behaviors in meal assistance.

FIG. 44 is an example of support information that is output in theexcretion assistance where the caregivers assist the assisted person toexcrete. Note that the excretion assistance may be performed in thetoilet or by using a diaper, Number 66-72 represent the output data ofthe excretion assistance in the toilet and Number 73-75 represent theoutput data of the excretion assistance using a diaper.

FIG. 45 is an example of support information that is output in transferassistance and moving assistance where the caregivers assist theassisted person to transfer and move the assisted person. Thetransferring assistance and the moving assistance may vary in thepresence or absence of equipment or in the type of equipment, dependingon the condition of the assisted person and the availability of thelift. In the example of the FIG. 45 , Number 92-103 represents outputdata when the assistance is provided using a wheelchair, Number 104-107represents output data when the assistance is provided using a cane, andNumber 108-112 represents output data when the assistance is providedusing a lift.

As shown in FIGS. 43 to 45 , the support information may includeinformation that supports at least one of the meal assistance, theexcretion assistance, and the transferring or moving assistance. In thisway, it becomes possible to appropriately support the assistance that ishighly necessary in the nursing homes, etc. For example, by supportingthe meal assistance, it is possible to control incidents such asaspiration and to improve the nutritional status of the assisted person.By supporting the excretion assistance, it becomes possible to controlthe excretion leakage, reduce the number of man-hours to deal with theexcretion leakage and the risk that occurs, reduce the excretiondisorder, and reduce the falling risk. In addition, by supporting thetransferring and moving assistance, it is possible to reduce the fallingrisk and to prepare the need for the assisted person in advance.

2.3.2 Sample Configuration of NN for Support Information Output

FIGS. 10 to 12 show the detailed configuration examples of the supportinformation output NN 122 shown in FIG. 9 . As shown in FIG. 10 , thesupport information output NN 122 may be a set of multiple NNs in whicheach NN outputs one support information. The support information 1 inFIG. 10 corresponds to any one of the support information shown in FIGS.43 to 45 . The input data group 1 represents one or more input datarequired to output the support information 1 among the multiple inputdata shown in FIGS. 28 to 42 . The same applies to the supportinformation 2 and later.

Also, as shown in the FIG. 11 , the support information output NN 122may a set of multiple NNs that can collectively output a number ofoutput data. In the example of FIG. 11 , the support information outputNN 122 includes a meal assistance support information output NN, aexcretion assistance support information output NN, and a transferringand moving assistance support information output NN.

For example, a meal assistance support information output NN may outputmultiple meal assistance support information. The output data of themeal assistance support information output NN corresponds to themultiple support information shown in the FIG. 43 . The input data ofthe meal assistance support information output NN represents themultiple input data required for outputting the meal assistance supportinformation among the multiple input data shown in the FIGS. 28 to 42 .The output of the excretion assistance support information output NNcorresponds to the multiple support information shown in the FIG. 44 .The output of the transferring and moving assistance support informationoutput NN corresponds to the multiple support information shown in FIG.45 .

Also, as shown in the FIG. 12 , the support information output NN 122may consists of 1 NN. The input data of the NN in FIG. 12 is the set ofall the data shown in FIGS. 28 to 42 , and the output data is the set ofall the support information shown in FIGS. 43 to 45 .

Also, the configuration of the support information output NN 122 is notlimited to the FIGS. 10 to 12 . For example, by dividing the mealassistance support information output NN into several pieces, theintermediate configuration of the FIG. 10 and the FIG. 11 may be used.In addition, the specific configuration of the support informationoutput NN 122 can be modified in various ways.

FIG. 13 shows an example of the relationship between the factordetermination NN 121 and the support information output NN 122. Theinput data for factor determination in FIG. 13 is the input in the FIG.6 and includes dementia level information, etc. The input data of thesupport information output NN 122 is the input in the FIG. 9 ,specifically, the data shown in FIGS. 28 to 42 . Some input data forfactor determination and input data for support information output mayoverlap.

In the example shown in the FIG. 13 , the output data of the factordetermination NN 121 is used as part of the input data of the supportinformation output NN 122. The output data of the factor determinationNN 121 may be information to identify one factor that is thedetermination result as described above or may be information based onmultiple probability values. As shown in the FIG. 10 or the FIG. 11 ,when the support information output NN 122 includes multiple NNs, theoutput data of the factor determination NN 121 may be input to all NNsor to some NNs. In this way, the support information can be output basedon the result of the factor determination in the factor determinationunit 111. As a result, it is possible to make the caregiver understandthe degree of dementia progression of the assisted person, and toprovide various kinds of assistance according to the degree of dementiaprogression.

If there is no abnormal behavior in the assisted person, the output ofthe factor determination NN 121 may be treated as 0. Also, as describedabove, the factor determination NN 121 may be capable of outputtinginformation indicating “no abnormal behavior.”

However, in the method of the present embodiment, the determinationresult by the factor determination unit 111 may be used to output thesupport information, and the specific method is not limited to theexample in the FIG. 13 .

2.3.3 Learning Processing and Inference Processing

The flow of learning processing of the support information output NN 122by the learning unit 114 is the same as the flow for creating the factordetermination NN 121. The training data for creating the supportinformation output NN 122 includes a data set in which correct answerdata representing assistance results performed by a skilled caregiverusing tacit knowledge is associated with input data.

For example, in the case of the meal assistance to an assisted person,at least the sensors required for the meal assistance are turned onamong the sensor group 400. As a result, among the input data shown inthe FIGS. 28 to 42 , data related to the meal assistance is acquired bythe sensor group 400 and stored in the storage unit 120 of the serversystem 100. In addition, data representing the results of the assistanceof the caregiver, such as the posture the skilled caregiver made theassisted person take (corresponding to Number 9-12 in the FIG. 43 ,etc.), the timing of serving the meal with the spoon (corresponding toNumber 26 in the FIG. 43 ), and the amount served per bite(corresponding to Number 25 in FIG. 43 ), are stored in the storage unit120 as correct data.

The learning unit 114 obtains the output data by inputting the inputdata among the training data into the support information output NN 122and by performing a forward operation using the weights at that time.Moreover, the learning unit 114 obtains an objective function (e.g., anerror function such as a mean squared error function) based on theoutput data and the correct answer data, and updates the weight so thatthe error is reduced by using the backpropagation method or the like.The support information output NN 122 at the end of learning is storedin the storage unit 120 as a learned model.

As shown in the FIG. 13 , the output of the factor determination NN 121may be included in the input of the support information output NN 122.In this case, the input data in the training data includes theinformation representing the factors of the abnormal behavior of theassisted person. For example, as described in the learning process ofthe factor determination NN 121, the correct answer data imparted by anexpert such as a physician may be used as one of the input data in thetraining data. Alternatively, when the learning processing of the factordetermination NN 121 has already been completed, the inferenceprocessing using the factor determination NN 121 may be performed, andthe result may be used as one of the input data in the training data, asshown in FIG. 8 .

The correct answer data, as in the example above, is information thatrepresents the results of assistance performed by a skilled caregiverusing implicit knowledge. It is possible for a skilled caregiver toprovide assistance that is naturally appropriate for the assistedperson, considering the degree of dementia progression of the assistedperson, etc. Thus, it is possible to machine learn appropriateassistance according to the factors of abnormal behavior by using theresults of assistance of skilled caregivers as correct answer data. Theprocessing after acquiring the training data is the same in this case.That is, the learning unit 114 performs the forward operation using theinput data in the training data, obtains an error function from theoutput data and the correct answer data, and updates the weights tominimize the error.

The support information output unit 112 of the server system 100acquires the input data shown in the FIGS. 28 to 42 in the inferencestage. The input data here may include data that can output the desiredsupport information, and it is not essential to acquire all the inputdata in the FIGS. 28 to 42 . Also, the support information output unit112 acquires the determination result of the factor determination unit111 as one of the input data. The support information output unit 112reads the learned support information output NN 122 from the storageunit 120 and inputs the input data into the support information outputNN 122. As shown in the FIGS. 11 and 12 , in case of that a NN capableof outputting multiple pieces of support information is used and onlysome of the support information is required to be output, some of theinput data may not have been acquired. In this case, the supportinformation output unit 112 may, for example, set the value ofunacquired input data to 0. The support information output unit 112obtains support information as output data by performing forwardoperation.

3. Processing Flow

Next, the flow of detailed processing to support the assistance of theassisted person by the caregivers in the assisted living facility, etc.is explained.

3.1 Factor Estimation of Behavior

First, the server system 100 determines whether or not abnormal behavioris observed in the assisted person, and if abnormal behavior isobserved, what is the factor of the abnormal behavior, apart fromprocessing for initiating and executing a detailed assistance sequence.

For example, the factor determination unit 111 periodically performs theprocessing described above using the FIG. 8 . In this way, the presenceor absence of abnormal behavior and the factors of abnormal behavior canbe determined for each of the multiple assisted persons who areassistance targets. In the following explanation, it is assumed that theresult of the factor determination by the factor determination unit 111has been obtained.

3.2 Assisted Support

3.2.1 User Settings

The examples of the support information in this embodiment are shown inFIGS. 43 to 45 . The support information output unit 112 may output allthis support information. However, if too much information is reported,inexperienced caregivers may not be able to grasp the content or not beable to recognize the difference in importance from process to process.Also, since the caregivers with some experience can properly perform aprescribed assistance without any support, the notification of supportinformation may make the caregivers with some experience annoying.Therefore, in this embodiment, the users who are the caregivers may beable to set the support information to be output.

The storage unit 120 of the server system 100 may store the firstassociation information 123. The FIG. 14 is a detailed example of thefirst association information. As shown in the FIG. 14 , the firstassociation information 123 is information that associates a caregiverID that identifies the caregiver with support information andinformation that represents the output setting of the supportinformation.

The output settings include active and inactive. When the prescribedsupport information is set to be active, the support information outputunit 112 outputs the support information to the subject caregiver. Whenthe prescribed support information is set as inactive, the supportinformation output unit 112 does not output the support information tothe subject caregiver. In this way, the support information to be outputcan be flexibly set for each caregiver.

However, as shown in FIGS. 28 to 42 , there are so many types of inputdata assumed in this embodiment, and the nursing homes may not be ableto obtain all the input data. For example, due to constraints such asbudgets and the structure of nursing homes, it may not be possible todeploy the necessary sensors to obtain the prescribed input data. Inthis case, by missing input data, it is possible not to obtain theprescribed support information with sufficient accuracy.

Thus, the support information output setting includes “can't output” inaddition to an “active” or an “inactive”. “can't output” indicates asetting that does not output support information because required inputdata could not be obtained. The “inactivity” is different from “can'toutput” because it represents the required input data can be obtainedbut the support information is intentionally not output.

For example, the storage unit 120 of the server system 100 may store thesecond association information 124 and the third association information125. The FIG. 15 shows a detailed example of the second associationinformation 124. As shown in the FIG. 15 , the second associationinformation 124 includes the support information and the required inputdata set required for the output of the support information. The supportinformation is any of the multiple data shown in the FIGS. 43 to 45 .The required input data group is one or more of the data shown in theFIGS. 28 to 42 . The required input data group may be, for example, dataspecified by the user. Alternatively, the support information output NN122 may be created for each of the multiple candidate input data groups,and the candidate input data group with the highest accuracy rate usingthe validation data may be selected as the required input data group. Inaddition, the required input data group is not limited to one set, andmultiple candidate input data groups with correct rate greater than apredetermined threshold may be used as the required input data group.

The FIG. 16 is a detailed example of the third association information125. The third association information 125 is information thatassociates nursing homes with the input data that can be obtained at thenursing homes. For example, a person in charge of a nursing home mayselect input data that can be obtained at the nursing home and transmitthe selection results to the server system 100. Alternatively, thesensor group 400 deployed in the nursing home transmits informationidentifying the nursing home to the server system 100 in associationwith the sensor information. The processing unit 110 of the serversystem 100 may create the third association information 125 based on theacquisition history of the sensor information.

The setting unit 113 of the server system 100 determines whether or noteach support information can be output for each nursing home based onthe second association information 124 and the third associationinformation 125. Specifically, the setting unit 113 determines whetheror not the support information can be output based on whether or not therequired input data group necessary for the output of the supportinformation is included in the input data group obtainable in the targetnursing home.

Also, there is a correspondence between the input data and the sensorused to acquire the input data. Therefore, the storage unit 120 maystore the fourth association information that associates the input datawith one or more sensors used to acquire the input data. By using thefourth association information in addition to the second associationinformation 124 and the third association information 125, it ispossible to determine whether or not the support information can beoutput on a sensor-by-sensor basis. Alternatively, the fourthassociation information may not be provided separately, instead, theinput data of the second association information 124 and the thirdassociation information 125 may be replaced with the information of thesensor.

Also, a device including a prescribed sensor is not limited to one. Forexample, if the motion sensor 410 and the imaging sensor 420 arerequired, a device such as a smartphone including both a camera and anacceleration sensor may be utilized, or two separate devices may beutilized.

Among the cameras, the multiple model cameras with differentresolutions, different magnifications, etc. are available. Therefore,the storage unit 120 may store a fifth association information thatassociates the sensor with the device including the sensor. In thiscase, the data can be managed on a device-by-device basis. For example,if a nursing home designates a device that has already been installed,the sensors included in the device and input data that can be obtainedusing the sensors are identified in the server system 1000. It ispossible to improve a user convenience because it is not necessary forthe person in charge of the nursing home or the caregiver to grasp thesensors included in the device or the input data that can be acquired bythe device.

An example of determining whether the support information can be outputon a nursing home basis is explained above (see, for example, the FIG.16 ). However, the method of this embodiment is not limited to this. Forexample, if a nursing home has a first space for residents with highlevels of care needs and a second space for residents with low levels ofcare needs, the first space may have many sensors and the second spacemay have fewer sensors. In this case, the server system 100 mayseparately manage the support information that can be output in thefirst space and the support information that can be output in the secondspace. In addition, various modifications can be made to the detailedmethods, such as managing whether or not the support information can beoutput for each assisted person.

The FIG. 17 shows an example of a setting screen for setting the supportinformation to be output. The processing described below is realizedthat, for example, the storage unit 212 of the mobile terminal device210 stores a Web application program communicating with the serversystem 100, and the processing unit 211 operates according to the Webapplication program. For example, displaying a display screen andaccepting user operations can be done according to the Web applicationprogram by using the display unit 214 or the operation unit 215. Inaddition, the setting unit 113 of the server system 100 performsprocessing to generate the display screen, update the display screen orcontrol the database according to the user operation. However, themethod of this embodiment is not limited to using a Web applicationprogram, and various modifications such as using so-called nativeapplications are possible. Although the FIG. 17 shows an example inwhich a setting screen is displayed on the display unit 214 of themobile terminal device 210, the setting screen may be displayed on ananother caregiver device 200.

For example, the setting screen is a screen in which “active”,“inactive”, and “can't output” can be selected for each of the multiplepieces of the support information. The FIG. 17 shows an example of asetting screen that includes objects OB1 to OB3 corresponding to threepieces of the support information: “timing for changing a diaper” whichis the support information supporting the assisted excretion, and“serving amount by the spoon” and “serving timing by the spoon” which isthe support information supporting the meal assistance.

For example, if the corresponding support information is “active”, theobject is displayed in the first mode. If the corresponding supportinformation is “inactive”, the object is displayed in a second mode. Ifthe corresponding support information can not output, the object isdisplayed in a third mode. The display mode may be controlled using thesize, shape and color of the object, or the size, font, color, etc. ofthe text included in the object. In addition, the detailed display modecan be modified in various ways.

In the FIG. 17 , objects OB1 to OB3 are buttons, and the colors of thebuttons are different depending on “active”, “inactive”, or “can'toutput”. For example, “timing for changing a diaper” is “active”,“serving amount by the spoon” is “can't output”, and “serving timing bythe spoon” is “inactive”. In this case, the support information outputunit 112 outputs the support information representing “timing forchanging a diaper” and does not output the support informationrepresenting “serving timing by the spoon”. In addition, in the nursinghome, since it may be difficult to accurately determine the “servingamount by the spoon” due to a shortage of sensors, the output of“serving amount by the spoon” is not acceptable to output. Since eachobject OB1˜OB3 is displayed in a different manner, it is possible topresent the current setting to the caregivers in an easy-to-understandmanner.

An operating the operating part 215 of the mobile terminal device 210 bythe caregivers can switch “active” or “inactive”. For example, when thecaregiver performs an operation to select “timing for changing adiaper,” information indicating this is transmitted to the server system100. The setting unit 113 performs processing for updating the outputsetting corresponding to the “timing for changing a diaper” of thetarget caregiver ID into “inactive” in the first association information123. The setting unit 113 also generates a display screen in which thecorresponding object OB1 is displayed in the second mode correspondingto the “inactive” one and transmits it to the mobile terminal device 210via the communication unit 130. The display unit 214 displays thedisplay screen.

Similarly, when an object selection operation corresponding to theinactive support information is performed, the setting unit 113 updatesthe output setting corresponding to the target caregiver and the targetsupport information into “active”. The display unit 214 changes thedisplay mode of the object in which the selection operation has beenperformed to the first mode.

On the other hand, even if an object selection operation correspondingto the support information that is “can't output” is performed, thedisplay unit 214 maintains the display in the third mode that represents“can't output”. In this case, the setting unit 113 does not performupdate processing of the first association information 123.

In addition, when an object selection operation corresponding to supportinformation that is “can't output” is performed, the input datanecessary for outputting the support information may be suggested. Forexample, the server system 100 may specify necessary input data based onthe second association information 124 and display the input data on thedisplay unit 214 of the mobile terminal device 210. Also, as mentionedabove, the input data described above may be replaced by a sensor or adevice. For example, the setting unit 113 may identify the sensor or thedevice required to output the support information for which theselection operation by the user has been performed, and may display thesensor or the device on the display unit 214 of the mobile terminaldevice 210.

The FIG. 18 is a flowchart explaining the above setting process.Firstly, the caregiver performs the setting change operation using hisor her caregiver device 200. In the step S301, the setting unit 113 ofthe server system 100 accepts the setting change operation via thenetwork NW.

In the step S302, the setting unit 113 performs processing to displaythe setting screen on the caregiver device 200 based on the firstassociation information 123 at that time and the caregiver IDrepresenting the caregiver who performed the setting change operation.The processing in the step S302 may be the processing of creating animage corresponding to the setting screen and transmitting the image tothe caregiver device 200, or the processing of transmitting informationfor generating the setting screen to the caregiver device 200. Theinformation for generating the setting screen may be an extracted resultextracted part of the data corresponding to the caregiver ID among thefirst association information 123. Also, the information for generatingthe setting screen may be the processing result in which some processingis performed on the extracted result. Thus, for example, a screencorresponding to the FIG. 17 is displayed on the display unit 214 of themobile terminal device 210.

In the step S303, the setting unit 113 determines the user operationperformed by the caregiver device 200. When no operation to change thesetting is detected, the setting unit 113 terminates the processing.

In the case that the setting operation to activate the inactive supportinformation is performed, or the setting operation to deactivate theactive support information is performed, the setting unit 113 reflectsthe setting change in the step S304. Specifically, the setting unit 113performs processing to update the first association information 123based on the information from the caregiver device 200.

Also, if a selection operation for the support information that is“can't output” is performed, at the step S305, the setting unit 113identifies the input data, the sensor, or the device that is missing forthe output of the support information. In the step S306, the settingunit 113 presents and proposes the identified input data, or theidentified sensor, or the identified device to the caregivers. Theprocessing in the step S306 may be processing to transmit the displayimage itself or processing to transmit information used to generate thedisplay image, similar to the processing in the step S302. Thepresentation described here is not limited to display, and thepresentation processing using the voice or the like may be performed.

3.2.2 Output Processing of the Support Information

The FIG. 19 is a flowchart illustrating an output processing of thesupport information by the support information output unit 112. In thestep S401, the support information output unit 112 acquires the inputdata corresponding to the support information to be output.Specifically, the storage unit 120 of the server system 100 stores oneor more input data to output the target support information among themultiple input data shown in the FIGS. 28 to 42 . The supportinformation is associated with the input data using, for example, theaforementioned second association information 124.

In the step S402, the support information output unit 112 obtains thesupport information by inputting necessary input data into the supportinformation output NN 122.

In the step S403, the support information output unit 112 determineswhether or not a notification based on the support information isnecessary. If the notification is necessary, the support informationoutput unit 112 performs the notification processing in the step S404.The notification may be a voice notification using a headset earphone orthe like, a display using the display unit 214 of a mobile terminaldevice 210, or other notification. When the notification is unnecessaryor after the notification processing is performed, the supportinformation output unit 112 terminates the processing.

As noted above, the number of the support information items to be outputcan vary depending on the type of sensors installed in the nursing homeand the settings of the caregiver. However, even in both cases, the flowof processing shown in the FIG. 19 for each piece of the supportinformation to be output is the same, the flow of processing includesthe identification of the input data, the operation by NN, and thenotification if necessary.

If the processing performance of the server system 100 is sufficient,the support information output unit 112 will always perform theprocessing shown in the FIG. 19 about all the support information set asoutput targets, and appropriately perform the notification processingabout items which need the notification.

Also, considering the reduction of processing load, the processing shownin the FIG. 19 may then be performed about the limited supportinformation which is needed at that time. For example, as shown in theFIGS. 43 to 45 , the support information can categorize the neededsituations, such as the support information needed for the mealassistance, the support information needed for the excretion assistance,and so on. Accordingly, the support information output unit 112 mayidentify the necessary support information in the current situation andexecute the processing shown in the FIG. 19 for the identified supportinformation. For example, the support information output unit 112 maydetermine whether the assistance is to be started for the mealassistance, the excretion assistance, and the transferring or movingassistance, respectively. The support information output unit 112performs the processing shown in the FIG. 19 for the support informationrelated to the assistance determined to be started. The startdetermination is described later with reference to the FIG. 20 .

In addition, in the case of the meal assistance, it is possible toclassify the assistance into “the assistance before the meal”, “theassistance during the meal”, “the assistance after the meal” inchronological order. Therefore, the support information output unit 112can specify the executed order in which the processing shown in the FIG.19 among the multiple pieces of the support information. In addition,depending on the assistance, there may be restrictions on the order ofexecution and the need for execution among the assistance, such as thefact that the second assistance is required only when the firstassistance is performed.

Therefore, the support of the assistance by the information processingsystem 10 of this embodiment may be performed according to assistancesequences that combine a plurality of assistance. Specifically, theassistance of the assisted person by the caregiver is supported by thesupport information output unit 112 sequentially outputting the multiplesupport information according to the assistance sequences.

The following describes examples of the assistance sequences for themeal assistance, the excretion assistance, and the transferring ormoving assistance. Specifically, the start determination of eachassistance sequence is determined firstly, and then the specific flow ofeach assistance sequence is explained.

As described later with reference to the FIGS. 21 to 23 , for the sakeof illustrative convenience, only some of the support information in theFIGS. 43 to 45 is output as a target in the following assistancesequence. However, it is easy for those skilled person in the art tounderstand that in each assistance sequence described below, variationssuch as omitting the output of some support information or adding theoutput of other support information shown in FIGS. 43 to 45 arepossible.

3.2.3 Start Determination

The assistance in this embodiment may include the meal assistance, theexcretion assistance, and the transferring or moving assistance.However, these assistance need not be performed all the time, and theassistance sequence is performed when the assisted person needs theassistance and there exists the caregivers who can perform theassistance. That is, in the present embodiment, the start determinationof the assistance sequence is performed firstly, and whether thestarting or waiting of the assistance sequence may be determinedaccording to the result of the start determination.

FIG. 20 is a flow chart to explain the start determination. This processis periodically performed, for example for each assisted person. First,in the step S501, the support information output unit 112 acquires atleast a part of the input data shown in FIGS. 28 to 42 . In the stepS502, the support information output unit 112 obtains the supportinformation by inputting the acquired input data into the supportinformation output NN 122. Here, the support information is theinformation to identify at least one of the following: a timing to startthe meal assistance, a timing to start the excretion assistance, and atiming to start the transferring or moving assistance. For example, thesupport information output unit 112 may determine whether or not eachassistance should be started at the timing when the processing shown inthe FIG. 20 is performed. Alternatively, the support information outputunit 112 may output the information specifying a detailed time, such aseach assistance should be started a certain number of minutes later.

In the step S503, the support information output unit 112 determineswhether the current timing would be a timing of the assistance sequenceto start. If it is determined that the current timing is not the timingto start, the support information output unit 112 completes theprocessing and waits again until the processing shown in the FIG. 20 isperformed.

For example, the support information output unit 112 determines a timingof the meal assistance sequence to start by using as input data, thecomplexion of each assisted person, the body temperature of eachassisted person, the body weight of each assisted person, the medicationof each assisted person, the history of past meals of each assistedperson, the history of excretion of each assisted person, the history ofrehabilitation of each assisted person, etc., in addition to theinformation of the meal schedule in the nursing homes.

Also, regarding to the excretion assistance, we consider a case thatroughly schedules such as five times a day are decided. Therefore, thesupport information output unit 112 determines a timing of the excretionassistance sequence to start by using as input data, the amount andtiming of meals for each assisted person, the amount and timing of waterintake, whether or not laxatives have been administered, the pasthistory of the excretion, the rehabilitation record, the pressure ulcercondition, etc., in addition to the information of the excretionassistance schedule in the nursing home.

Also, the support information output unit 112 determines a timing of theassistance sequence related to the transferring or moving assistance tostart by using as the input data, the ADL of the assisted person, themedical history of the assisted person, etc., in addition to whether theevents requiring the transfer of the assisted person would occur, suchas the meals and the recreation.

When it is determined that the current timing is the timing of theassistance sequence to start, in the step S504, the support informationoutput unit 112 performs processing to determine a caregiver to assist atarget assisted person. For example, the support information output unit112 holds the information such as the work schedule of the caregivers inthe nursing home and the assignment of the assisted person, and maydesignate a caregiver based on that information.

In the step S505, the support information output unit 112 performs thenotification processing to instruct the start of the assistance sequenceto the caregiver device 200 of the designated caregiver. For example,the support information output unit 112 may perform the processing toreproduce a voice such as “Please start the meal assistance for Mr. A”in wearable device 220 such as a headset. The support information outputunit 112 may also perform the processing to display the same text on thedisplay unit 214 of the mobile terminal device 210.

In the step S506, the support information output unit 112 determines theresponse of the caregiver to the above notification processing. Forexample, three responses may be set: “OK,” “later” and “transfer” as theresponse of the caregiver. The caregiver may respond by voice. Thecaregiver's response may be obtained based on detection results usingthe microphone on the headset, for example. The caregiver's response mayalso be realized by other ways such as the text input.

A “OK” means a response indicating that the instructed assistancesequence can be started. In this case, in the step S507, the supportinformation output unit 112 transfers to a detailed assistance sequence.For example, the support information output unit 112 starts theprocessing of the FIG. 21 , FIG. 22 , and FIG. 23 , etc.

A “Later” is a response indicating that the assistance sequence can notbe started immediately, but is considered ready to start after aprescribed period of time. For example, it may be the case that thecaregiver is currently engaged in another task, but the instructedassistance sequence can be started once the task is completed. In thiscase, in the step S508, the support information output unit 112 waitsfor a prescribed time, and after waiting, returns to the step S505 andexecutes the notification processing to the same caregiver again.

A “transfer” is a response indicating that it is difficult for thecaregiver to perform an assistance sequence and the caregiver asks for arequest to an another caregiver. In this case, the support informationoutput unit 112 returns to the step S504 and select an anothercaregiver. The processing after the step S505 is the same.

However, a processing when “transfer” is selected is not limited tothis. For example, if the designated caregiver selects “transfer,” thesupport information output unit 112 may simultaneously notifynotifications to multiple caregivers. Then, among the multiplecaregivers, the caregivers who responded “OK” may be selected and thespecific assistance sequence may be started for the caregivers.

3.2.4 The Meal Assistance

FIG. 21 is a flowchart illustrating a detailed assistance sequence whenthe meal assistance is provided.Firstly, when the meal assistance sequence is started, in the step S601,the support information output unit 112 controls to turn on the sensorsnecessary for the meal assistance support among the sensor groups 400arranged in the nursing home, etc. In the step S601, the supportinformation output unit 112 may remotely control to switch “on” or “off”of the sensors in the sensor group 400. Or the support informationoutput unit 112 may designate a sensor or a device and send a message toa device in a nursing home such as the caregiver device 200 to make thecaregiver turn on the sensor or the device. Although this is notexplicitly shown in the flowchart, the sensor group 400 periodicallytransmits the sensor information to the server system 100, and thesupport information output unit 112 can acquire the input data necessaryfor outputting the support information.

In the step S602, the support information output unit 112 outputs thesupport information for providing the meals according to the assistedperson based on the support information output NN 122. For example, inthe step S602, the support information output unit 112 outputs thesupport information to instruct the meal according to the allergy of theassisted person and the medication according to the medical condition.

Next, in the step S603, the support information output unit 112determines whether the assisted person and the caregiver have moved to aposition where the assisted person will eat. The Meals may be served inthe assisted person's room or in the dining room. The processing in thestep S603 is performed by taking as input data, the information that canidentify the location of the assisted person and the caregiver using acamera or radio frequency identifier (RFID) etc. In the processing ofthe step S603, for example, the support information output unit 112 maydetermine that the assisted person and the caregiver have moved to theposition where the assisted person eat if the image of the assistedperson and the meal is captured on the screen of the camera carried bythe caregiver.

If at least one of the assisted person and the caregiver is not in theposition, in the step S604, the support information output unit 112waits for a certain time and then performs the processing of the stepS603 again.

If the assisted person and the caregiver are in position, in the stepS605, the support information output unit 112 obtains the minimum amountof the meal to eat. The minimum amount to eat here may be less than theamount of the served meal. In other words, the caregiver does not haveto feed all the served meals and does not have to force the assistedperson to eat more once the minimum amount is reached. In the processingof the step S605 is performed using, as the input data, the assistedperson's complexion, the care record of the assisted person, the weightchange of the assisted person, the meal schedule of the assisted person,etc.

In the step S606, the support information output unit 112 notifies thecaregiver of the required minimum amount to eat. The notification may beaudible notification using earphones such as a headset, or may bedisplayed using the display unit 214 of a mobile terminal device 210.

In the step S607, the support information output unit 112 determines atiming of serving a meal with a spoon and the amount served with thespoon.

The timing of serving a meal with a spoon represents the timing when onebite amount of the meal on the spoon is fed into the assisted person'smouth. The amount served with the spoon represents one bite amount ofthe meal. The processing of the step S607 is performed, for example, byusing the input data related to the chewing state of the assistedperson. The input data related to the chewing state is, for example, theinformation on the condition of the mouth of the assisted person, thecondition of the throat of the assisted person, the facial expression ofthe assisted person, the complexion of the assisted person, the postureof the assisted person, the changes of the meal in response to a talk,the timing of swallowing of the assisted person, a time period ofputting the meal in his or her mouth of the assisted person, the eatingrhythm of the assisted person, etc., and may be, for example, an imagedimage of the assisted person. In addition, the input data related to thechewing state is the information related to the movement of the jaw, themovement of the cheek, the movement of the whole face, and the movementof the body, and may be the sensor information of the motion sensor 410,for example. In addition, the input data related to the chewing statemay include a voice data indicating the quality of voice and volume ofvoice in response to a talk or the like during a meal eating, and theinformation indicating the difference in a timing and an amount duringthe meal eating in the past, as well as a seasonal difference and thedifference of the physical conditions. The information representing theeating rhythm may be an imaged image or the sensor information of themotion sensor 410. In addition, the sensors used to acquire the aboveinformation can be modified in various ways.

In the step S608, the support information output unit 112 informs thecaregiver of the timing of serving the meal with the obtained spoon andthe amount served with the spoon.

For example, in the step S607, the support information output unit 112determines whether the current timing is the timing of serving a mealwith a spoon. If it is determined that the current time the timing ofserving a meal with a spoon, the support information output unit 112notifies that in the step S608 and if it is determined the current timeis not the timing of serving a meal with a spoon, the supportinformation output unit 112 doesn't notify that in the step S608. Inaddition, in the case that it is determined that the current timing isnot the timing of serving, the support information output unit 112 mayreport that the current timing is not the timing of serving when thecaregiver intends to provide the meal to the assisted person.

Also, when it is determined that current timing is the timing ofserving, for example, the support information output unit 112 may obtainthe amount served with the spoon in the step S607 and inform thecaregiver of the requested amount in the step S608 using the number ofgrams or a step such as “more”, “less”, or “normal”. Alternatively, inthe step S607, the support information output unit 112 may obtain thesupport information representing whether the amount served with thespoon is appropriate by using the input data representing the mealamount actually placed on the spoon by the caregiver. The input data inthis case includes, for example, the output of a camera that images thehand of the caregiver. If the meal amount served with the spoon is toomuch or too little, in the step S608, the support information outputunit 112 may issue a notification urging to change the meal amountserved with the spoon.

The facial expression of the assisted person may be used to determinewhether the assistance of the assisted person is correct. For example,the support information output unit 112 may not output the instructionspecifically as a correct answer when the assisted person is smiling,but may output the instruction when the assisted person has andispleased face. For example, the support information output unit 112may use, as input data in the step S607, an image imaged the face of theassisted person or the result of the facial expression determinationprocessing based on the image. In this way, based on the facialexpression of the assisted person, it can be determined whether the paceof the meal serving is appropriate. Alternatively, the supportinformation output unit 112 may determine the degree of relaxation fromthe heart rate (pulse) analysis. The support information output unit 112may not output instructions specifically as correct answers when thedegree of relaxation of the assisted person is high, and may outputinstructions when the degree of relaxation of the assisted person islow. For example, the support information output unit 112 uses theinformation representing the heart rate, the pulse rate, or theiranalysis results as the input data in the step S607. Also, the stepsother than S607 in the FIG. 21 , the FIGS. 22 and 23 , which will bedescribed later, may use that the facial expression and degrees ofrelaxation may be applied in determining whether the assistance isappropriate or not.

In the step S609, the support information output unit 112 determineswhether or not the assisted person finishes eating the meal. If not, thesupport information output unit 112 returns to the step S607. Byrepeating the processing of the steps S607 and S608 in this way, itbecomes possible to present to the caregiver the timing of serving onebite of the meal and the amount of the meal to be served at that time,piece by piece. As a result, it is possible for the assisted person toeat the meal at an appropriate pace.

When it is determined that the meal is finished, in the step S610, thesupport information output unit 112 instructs the recording of the mealresult. For example, as the meal result, an image including the leftoverof the meal is acquired. It should be noted that the recordinginstruction may be one that instructs the caregiver to take an imageusing a mobile terminal device 210 or the like, or it may be one thatautomatically takes an image by remotely controlling a camera placed atan appropriate position.

In the step S611, the support information output unit 112 obtains thesupport information indicating whether the assisted person needs to behydrated. When the hydration is necessary, in the step S612, the supportinformation output unit 112 notifies the caregiver to instruct thehydration. The support information output unit 112 may obtain thesupport information representing a detailed amount of the replenishmentof the hydration in the step S611 and report the amount of thereplenishment of the hydration in the step S612.

If hydration is not necessary, or after performing the processing of thestep S612, the meal assistance sequence is completed.

FIG. 21 is an example of a meal assistance sequence, and the detailedsequence allows various modifications to be performed. For example, whenthe meal assistance is provided in a nursing bed 310, a controlprocessing to switch a meal mode suitable for the meal at the nursingbed 310 (e.g., a mode in which the back bottom is raised to a set anglein the range of 30 to 90 degrees, a mode in which the knee bottom israised to a set angle in the range of 0 to 30 degrees, a mode in whichthe foot bottom is lowered to a set angle in the range of 0 to 90degrees, a bed tilt angle to a set angle ranging from 0 degrees to 20degrees so that the head is higher) may be added. For example, thesupport information output unit 112 may request the support informationindicating whether the assisted person and the served meal are in asuitable condition for starting the meal by using the output of a cameraor the like as the input data. When it is determined that the assistedperson and the served meal is set appropriately, the support informationoutput unit 112 performs a notification processing to ask the caregiverswhether it is acceptable to move the nursing bed 310. The nursing bed310 may be changed to a meal mode if the caregiver responds “OK”.

In addition, the support information output unit 112 can output varioussupport information related to the meal assistance for an assistant or achef who is in charge of cooking until the meal is served in the diningarea.

3.2.5 The Excretion Assistance

FIG. 22 is a flowchart illustrating the detailed assistance sequencewhen the excretion assistance is done. When the excretion assistancesequence is started, in the step S701, the support information outputunit 112 controls to turn on the sensors necessary for the excretionassistance among the sensor groups 400 arranged in the nursing home,etc.

Next, in the step S702, the support information output unit 112determines whether the caregiver has moved to a position to assistexcretion. For example, if the assisted person excretes into a diaper onthe nursing bed 310, the excretion assistance is assumed to be done inthe assisted person's room. In this case, the processing of the stepS702 is carried out by taking as input data, for example, the output ofa camera arranged in a living room or the output of a camera of a mobileterminal device 210 carried by a caregiver, the output of RFID, etc.

When the caregiver is not in the position, in the step S703, the supportinformation output unit 112 waits for a prescribed time period and thenperforms the processing of the step S702 again. When the caregiver is inthe position, in the step S704, the support information output unit 112requests the support information regarding to remove the diaper. In thestep S705, the support information output unit 112 notifies therequested support information.

For example, when removing the diaper, if the posture of the assistedperson, the posture of the caregiver, the direction in which the diaperis removed, etc. are not appropriate, it is not desirable that fecesadhere to the clothes and sheets of the assisted person. Therefore, thesupport information output unit 112 may determine, for example, in thestep S704, whether the movement of the caregiver in removing the diaperis appropriate. For example, the support information output unit 112 mayacquire the motion that is the correct answer and compare the motionwith the actual motion of the caregiver. If it is determined to beinappropriate, in the step S705, the support information output unit 112may notify that it is inappropriate or may concretely instruct theappropriate movement.

In the step S706, the support information output unit 112 requests thesupport information related to how to wear the diaper. In the step S707,the support information unit 112 performs a notification processing ofthe requested support information.

The support information output unit 112 may determine, for example, inthe step S706, whether the movement of the caregiver when putting on thenew diaper is appropriate. For example, the support information outputunit 112 may acquire the motion that is the correct answer and comparethe motion with the actual motion of the caregiver. If it is determinedto be inappropriate, in the step S707, the support information outputunit 112 may notify that it is inappropriate or may concretely instructthe appropriate movement.

It is true there is a problem that the sheets, etc., get dirty when thediaper is removed, but it is easy for the caregiver to recognize thedirt and to deal with it relatively easily because the caregiver is byhis or her side. On the other hand, in the case of a fecal leak due toinadequate and inappropriate wearing, there is not always the caregiversnearby at the time of the fecal leak. In addition, considering theburden on caregivers, it is not easy to excessively increase thefrequency of excretion assistance, and there is a risk that fecalleakage may be left for a long time. In view of the above, the supportinformation output unit 112 may set conditions so that the notificationin the processing of the step S707 becomes easy to perform compared tothe processing of the step S705. For example, the threshold in the stepS707 is set to be smaller than the threshold in the step S705 in thecase that the notification in the steps S705 and S707 is performed ifthe degree of divergence between the correct answer and the actualmovement exceeds the threshold.

Alternatively, in the step S706, it may be determined in detail whetherthe wearing condition is appropriate by using the sensor information ofthe sensor provided on the diaper as the input data. Again, it ispossible to have them provide the assistance with a greater emphasis onwearing diapers.

In the step S708, the support information output unit 112 instructs therecording of the excretion state. Specifically, the support informationoutput unit 112 gives an instruction to take an image about the state ofthe urine and the stool and an instruction to measure the weight of theurine and the stool. The weight measurement may be the weightmeasurement of the diapers or the weight measurement of the garbage ifthe garbage is to be transported. The instructions for recording heremay be to have the caregivers perform imaging and weighing, or toremotely control the camera or the sensor.

FIG. 22 is an example of the excretion assistance sequence, and thedetailed sequence can perform various modifications. For example, whenthe excretion assistance is done on the nursing bed 310, a controlprocessing to change the height of the nursing bed 310 to a heightsuitable for the excretion assistance (For example, the height from thefloor to the top of the bottom is 50 mm to 100 mm without the need forthe caregiver to bend over) may be added. For example, in the case thatthe caregiver responds “OK” in the step S506 in the FIG. 20 , a state inwhich the caregiver can easily assist with excretion is realized bychanging the height of the nursing bed 310 when the caregiver arrives.If the nursing bed 310 has a speaker, the support information outputunit 112 may be controlled to output a voice to explain the purpose tothe assisted person before changing the height. The support informationoutput unit 112 may also inform the caregiver that the height of thenursing bed 310 has been changed.

3.2.6 The Transferring or Moving Assistance

FIG. 23 is a flowchart illustrating the detailed assistance sequencewhen the transferring or moving assistance is performed. When thetransferring or moving assistance sequence is started, in the step S801,the support information output unit 112 controls to turn on the sensornecessary for the support of the transferring or moving assistance amongthe sensor groups 400 arranged in the nursing home, etc.

Next, in the step S802, the support information output unit 112determines whether a lift is necessary for the transferring or movingassistance of the assisted person. The processing in the step S802 isperformed by taking as input data the body size difference between thecaregiver and the assisted person, the ADL of the assisted person, thetime required for transferring or moving, the inventory of lifts in thenursing home, etc.

If the lift is not required, the caregiver would manually transfer theassisted person to the wheelchair. In the step S803, the supportinformation output unit 112 obtains the support information about thetransferring by manual. In the step S804, the support information outputunit 112 performs the notification processing of the obtained supportinformation. Note that the support information output unit 112 may issuea notification asking the caregiver whether the wheelchair needs to belocked before transferring. When the caregiver answers that it isnecessary, the support information output unit 112 controls to lock thewheelchair. Alternatively, the support information output unit 112 mayautomatically determine whether the lock is necessary, and if it isdetermined to be necessary, instruct the caregiver to lock thewheelchair before the processing of the step S803.

The support information output unit 112 may determine, for example, inthe step S803, whether the usage of the caregiver's body in transferringby manual is appropriate. For example, the support information outputunit 112 may acquire the correct movements, such as the posture of thecaregiver, the position in which the assisted person is placed on thefoot, and compare the correct movements with the actual movements of thecaregiver. The caregiver's movements may be detected using the motionsensor 410 or the imaging sensor 420. In addition to the movement of thecaregiver, these sensors may also detect the movement and posture of theassisted person, because the positional relationship between theassisted person and the caregiver is important in the transferring ormoving assistance. If it is determined to be inappropriate, in the stepS804, the support information output unit 112 may notify that it isinappropriate or may concretely instruct the appropriate movement.

If a lift is required, in the step S805, the support information outputunit 112 controls to move the lift to the location of the assistedperson. In the step S806, the support information output unit 112obtains the support information relate to the transfer using the lift.In the step S807, the support information output unit 112 performs anotification processing of the obtained support information.

The support information output unit 112 may determine, for example, inthe step S806, whether a usage of the lift is appropriate. For example,a support information output unit 11 2 may acquire correct answer data,such as the wearing state of a sling that can safely hoist the assistedperson, and compare the correct answer with the actual state. If it isdetermined to be inappropriate, in the step S807, the supportinformation output unit 112 may notify that it is inappropriate, or mayconcretely indicate the appropriate wearing condition.

After lifting the assisted person up by the lift, he or she may sit in awheelchair, or may be moved as is. The support information output unit112 may determine which option is used in consideration of the inventoryof the lift, the transfer destination, and the condition of the assistedperson, and notify it to the caregivers.

FIG. 23 is an example of a transferring or moving assistance sequence,in which various modifications can be performed. For example, a controlmay be added to change the height of the nursing bed 310 (For example, anursing bed has a height from the floor to the top of the bottom; aheight which feet are firmly attached when sitting on the bed 200 mm to500 mm for an assisted person who is able to stand, a height which isslightly higher than the wheelchair when transferring to the wheelchair200 mm to 500 mm, and a height which is slightly lower than a wheelchairwhen transferring from the wheelchair to the bed 200 mm to 500 mm, etc.)to a height suitable for the transferring or moving assistance. Sincethe detailed control is the same as for the excretion assistance etc.,the detailed explanation is omitted.

3.2.7 Detailed Changes in the Assistance in Response to the Factors

The detailed sequence of the meal assistance, the excretion assistance,and the transferring or moving assistance has been explained above.Although the explanation is omitted in the FIGS. 21 to 23 , each supportinformation may be obtained based on the presence or absence of theabnormal behavior or the factors of abnormal behavior. For example, asdescribed above using the FIG. 13 , the presence or absence of theabnormal behavior and the factor determination result of the abnormalbehavior are used as input data when obtaining support information.

For example, if the factor determination unit 111 determines dementia asa factor, the support information output unit 112 change the output ineach assistance sequence as if the dementia were progressing.

For example, in the meal assistance, the support information output unit112 outputs the information indicating the amount of the meal served perone bite (the amount served with a spoon) and the information indicatingthe timing of serving a meal for one bite (timing of serving a meal witha spoon) as the support information. In this case, the supportinformation output unit 112 may change at least one of the amount andthe timing of serving when the behavior is determined to be the abnormalbehavior of the dementia factor, compared with when the behavior isdetermined not to be the abnormal behavior of the dementia factor. Inthis way, it will be possible to change the pace of serving the mealsappropriately between the assisted person who has the dementia and theassisted person who does not have the dementia. For example, the supportinformation output unit 112 may provide a smaller amount or a slowertiming of serving. In this way, the pace of serving the meals can beappropriately managed when the dementia makes the assisted person morelikely to choke.

In addition, the support information output unit 112 outputs, as thesupport information, the information specifying the timing of theexcretion assistance to start, which is the timing to start theexcretion assistance. In this case, if the behavior is determined to bean abnormal behavior caused by the dementia factor, the timing of theexcretion assistance to start may be changed compared to when thebehavior is determined not to be an abnormal behavior caused by thedementia factor. In this way, it would be possible to initiate theexcretion assistance sequence at the appropriate time, depending onwhether the person has the dementia or not. For example, the supportinformation output unit 112 may make the timing of the excretionassistance to start earlier by adjusting the timing of the excretionassistance to start in this way, it is possible to make it easier tokeep a clean state even when the dementia makes it difficult for theassisted person to control the timing of the excretion.

Otherwise, if it is determined to be an abnormal behavior due to thefactors of the dementia, the support information output unit 112 maychange the support information to provide the following meal assistance.For example, the support information output unit 112 sets a highpriority for the following notifications when it is determined to be anabnormal behavior caused by the dementia. For example, the supportinformation output unit 112 notifies the following notifications in thecase of a dementia factor, and does not notify the followingnotifications in the case that there is no abnormal behavior or in thecase of other factors except for the dementia factor. Or the supportinformation output unit 112 notifies the following notificationsregardless of whether or not the factor of dementia is existed, but maycontrol to make the notifications more likely to notify when the factorof dementia is existed. For example, the support information output unit112 may increase the notification frequency in case of thedementia-related factors, or it may relax conditions in determining theneed for the notifications.

-   -   defecating before a meal so as to devote oneself to the meal    -   notifications whether the assisted person is getting enough        sleep or feeling well    -   after serving the meal, the caregivers make the observations        without any assistance to understand today's situation    -   setting up the environment, preparing dishes to make the        assisted person calm down, and preparing dishes he or she love    -   shaping into an easy-to-eat posture    -   making the adjustments, such as shifting a timing of serving the        meal if the assisted person do not eat the meal more    -   talking to them to understand that it is a meal and assisting        them for the first bite    -   talking to them and teaching them how to eat    -   hydrating so as not to dehydrate    -   Adjusting the amount served with a spoon and the pace at which        they eat to avoid choking    -   Increasing the amount of activity if they do not eat the meal        more, or adjusting the rhythm of the life

Also, if it is determined to be an abnormal behavior caused by thedementia, the support information output unit 112 may modify the supportinformation to provide the excretion assistance as follows:

-   -   notifications whether the assisted person is getting enough        sleep or feeling well    -   selecting and using appropriate pants, diapers, pads, etc.    -   In case of the toilet defecation, making sure water is flushed        in the toilet.    -   Taking countermeasures against falling because the assisted        person may go to the bathroom more often    -   Watching the timing of his or her excretion and talk to guide        the assisted person to the toilet    -   Watching the timing of his or her excretion to guide the        assisted person to the toilet or to change diapers because there        is a possibility of coprophilia.    -   assigning the compatible caregiver

Also, the support information output unit 112 may output the supportinformation related to sleep assistance. If it is determined that thebehavior is abnormal due to a dementia factor, the support informationoutput unit 112 may change the support information to provide the sleepassistance as follows.

-   -   Adjusting rhythm his or her life to keep his or her autonomic        nerves healthy    -   Exercising to increase the amount of activity during the day    -   watching for the abnormal behavior at night

When monitoring at night, for example, the sensor information of asensor to detect whether the assisted person is in the nursing bed or amonitoring sensor is used as the input data.

For example, the support information output unit 112 instructs thecaregiver who does not provide the meal assistance for breakfast toinstall the sensor during breakfast. In the case of performing theexercise, the support information output unit 112 may inform thecaregiver to suggest the recreation or the rehabilitation afterassisting the excretion of the assisted person in the daytime, forexample.

As described above with the reference to the FIG. 6 , the factordetermination unit 111 may determines whether there is an environmentalfactor or an excretory disturbance factor as the factors of the abnormalbehavior. For example, if it is determined that the behavior is abnormaldue to an excretory disorder factor, the support information output unit112 may change the support information to provide the followingassistance.

-   -   notifying of the addition of laxatives to serve the dinner    -   changing the contents of the meal (applicable to breakfast,        lunch, and dinner)    -   instructing to provide water after eating the meals    -   suggesting the recreation and the rehabilitation after the        excretion assistance in the daytime

The support information output unit 112 may not only instruct theaddition of laxative but also suggest a specific type of laxative anddosing time. For example, the support information output unit 112 maynotify the type of laxative by using the information indicating how manyconsecutive days the laxative is administered, the information about thedefecation interval, etc., as input data. In addition, the supportinformation output unit 112 may instruct the caregiver to remove thesensors other than the excretion sensor among the sensors arranged torespond to the dementia when the assisted person determined to be causedby the dementia is subsequently determined to be caused by impairedexcretion.

Also, if it is determined that the behavior is abnormal due to theenvironmental factors, the support information output unit 112 maymodify the support information so as to provide the followingassistance:

-   -   automatically rhythms controlling of the speakers or the        lighting so as to match previous environmental data without the        environmental factors.

By approaching the environment same as the previous environment beforethe abnormal behavior occurred in this way, it becomes possible toarrange the life rhythm of the assisted person. The caregiver may beable to change the settings, such as temporarily stopping theapplication of automatically controlling or not applying automaticallycontrolling. In addition, the support information output unit 112 mayinstruct the caregiver to remove the arranged sensor to respond to thedementia when the assisted person had determined to have caused thedementia is subsequently determined to have caused the environment.

Also, if the behavior is determined to be a factor of dementia, thesupport information output unit 112 may increase the type of the supportinformation to be output in comparison with the case in which theabnormal behavior is not detected. For example, the above supportinformation for “setting up the environment, preparing dishes to makethe assisted person calm down, and preparing dishes he or she love” isoutput when it is determined to be a dementia factor, but the abovesupport information may not output when it is determined to be otherfactors. In this case, for example, the input data of a temperaturesensor, a humidity sensor, a illuminance sensor, and a barometricpressure sensor may be used to determine a favorable environment for theassisted person.

Therefore, when the behavior is determined to be a factor of thedementia the support information output unit 112 is designed to increasethe type of the sensor information in comparison with the case in whichabnormal behavior is not detected. In this way, since the variety ofinput data increases, and it becomes possible to obtain the accuratelysupport information for providing the appropriate assistance for thedementia.

Also, the support information output unit 112 may determine whether thenewly sensors should be added based on the information to identify oneor more sensors that can be used and the sensor information to be addedif the behavior is determined to be a dementia factor. Here, the one ormore sensors that can be used are specifically located in the targetnursing home and are identified based on the third associationinformation 125 in the FIG. 16 . As described above using the FIGS. 14to 16 , it is possible not to output the prescribed support informationwith sufficient accuracy due to some types of sensors deployed innursing homes, and such support information may be set to “can'toutput”. Therefore, it may be difficult to appropriately output thesupport information which is suitable for the dementia in some nursinghomes even if the factor determination unit 111 determines that dementiais a factor. The information processing device may, for example,determine whether or not the additional sensors are needed and maysuggest the additional sensors or the devices including such sensors. Inthis way, it is possible to output the support information appropriateto the factors.

As described above, it is assumed that appropriate assistance will varydepending on the presence or absence of abnormal behavior and thefactors contributing to the abnormal behavior. According to the methodof this embodiment, the factor determination result of the behavior ofthe assisted person is used when supporting the assistance by thecaregivers. As a result, it is possible to have the caregiver providethe assistance that is more appropriate for the assisted person.

Specifically, it is possible to convert the tacit knowledge of a skilledcaregiver into data and have a less-skilled caregiver provide theappropriate assistance. For example, a less-skilled caregiver can assistas well as an experienced caregiver, improving reproducibility of theassistance. In addition, since variation in care assistance skills issuppressed and the organizational management is facilitated, incidentssuch as the falling of the assisted person are suppressed. As a result,the occurrence of vacancies associated with hospitalization and theoccurrence of overtime associated with the preparation of accidentreports can be reduced. Curbing the incidents also curbs caregivers frombecoming too risk-sensitive, which can reduce the stress andconsequently reduce to leave a job. In addition, that will also improvethe satisfaction of the caregivers and their families and improve theirquality of life by enabling caregivers to improve their skills and workenvironment.

The part or most of the processing of the information processing system10 of this embodiment, the server system 100 of this embodiment, thecaregiver device 200 of this embodiment, etc., may be realized by aprogram. In this case, a processor such as a CPU executes a program torealize the information processing system 10 of this embodiment or thelike. In detail, the program stored in the non-transitory informationstorage medium is read, and the read program is executed by a processorsuch as a CPU. Here, an information storage medium (a medium that can beread by a computer) stores programs, data, etc., and its function can berealized by an optical disk, HDD, memory (Card type memory, ROM, etc.),etc. A processor such as a CPU performs various processes of the presentembodiment based on a program stored in an information storage medium.That is, a program for making the computer function as a part of thisembodiment is stored in the information storage medium.

Also, the method of the present embodiment can be applied to aninformation processing method that determines whether the behavior ofthe assisted person is an abnormal behavior of the dementia factor basedon (1) the information on the dementia level of the assisted person and(2) at least one of the environmental information, the excretioninformation, and the sleep information of the assisted person, andoutputs the support information to support the assistance of theassisted person by the caregivers based on the determination result andthe sensor information that is a sensing result about the caregivers whoassist the assisted person or the assisted person.

4 Example of Modification <The Parallel Processing of MultipleAssistance Sequences>

Each assistance sequence described above in the FIGS. 21 to 23 may beperformed sequentially. For example, a designated caregiver performs asequence corresponding to any of the FIGS. 21 to 23 by responding “OK”in the step S506 of the FIG. 20 in a standby state, and returns to astandby state after completion. The standby state refers to a conditionin which the targeted caregiver has not performed any of the assistancesequences. Then, again, by responding “OK” in the step S506, thesequence corresponding to any of the FIGS. 21 to 23 is performed.

However, in the nursing homes, etc., one caregiver may assist severalassisted persons in parallel. For example, one caregiver simultaneouslyperforms the meal assistance for the assisted person A and the mealassistance for the assisted person B while the assisted person A and theassisted person B are seated in close positions. In this case, it isinefficient that the meal assistance sequence of FIG. 21 for theassisted person B is performed after completing the meal assistancesequence of FIG. 21 for the assisted person A.

Thus, the support information output unit 112 may be able to executemultiple assistance sequences in parallel with respect to one caregiver.For example, in the above example, the support information output unit112 executes the meal assistance sequence for the assisted person A andthe meal assistance sequence for the assisted person B in parallel. Inthis example, the ration of the caregiver and the assisted person are1:2, but the number of assisted persons in charge of one caregiver atthe same time may be three or more.

For example, the support information output unit 112 performs theprocessing of the step S605 in the meal assistance sequence of theassisted person A, and notifies the result in a form such as “Theminimum amount served for Mr. A is x grams” in the step S606. Similarly,in the meal assistance sequence of the assisted person B, the processingof the step S605 is performed, and the result is reported in a form suchas “The minimum amount served for Mr. B is y grams” in the step S606. Inthis way, the support information output unit 112 acquires the inputdata concerning the assisted person A and the input data concerning theassisted person B in parallel, and outputs the support informationconcerning the assisted person A and outputs the support informationconcerning the assisted person B at necessary timing based on therespective input data. In this way, even if there is a one-to-manyrelationship between the caregiver and the assisted person, it ispossible to have the caregiver perform the necessary assistance for eachassisted person. By installing a wide-angle camera capable ofsimultaneously imaging multiple assisted persons, it is also possible toshare the input data concerning the assisted person A with the inputdata concerning the assisted person B.

However, since there is only one caregiver, it is not easy to respond toall support information, if multiple support information is notified atvery close timing. For example, when the notification for the assistedperson A in the step S608 is notified, the caregiver picks up a spoonwith meal of the amount according to the notification and carries themeal to the mouth of the assisted person A. If the notification of thestep S608 is notified for the assisted person B before completing, it isdifficult for the caregiver to take the assisted person B's meal with aspoon and carry it to the assisted person B's mouth.

Additionally, when providing the meal assistance to more than oneassisted person, it is more efficient to let the assisted person eat themeal after everyone has gathered at a dining area such as a dining room.Therefore, even if it is determined that the caregiver and the assistedperson A are in the same position (Yes in the step S603), there may beun-desirable cases to start the processing such as the steps S607 toS609 for the assisted person A when the assisted person B is not in thesame position.

In view of these, the support information output unit 112 may not simplyperform the assistance sequence for the multiple assisted persons inparallel, but may also perform the processing considering therelationship between the multiple assistance sequences. For example,when multiple assistance sequences are performed in parallel for thedesignated caregiver, the support information output unit 112 maycontrol the execution and the stop (suspend) of each assistancesequence.

For example, the support information output unit 112 may suspend themeal assistance sequence regarding the assisted person B if thenotification for the assisted person A in the step S608 is notified, andresume the meal assistance sequence for the assisted person B when thecaregiver has finished to serve a bite of the meal for the assistedperson A. Since the support information output unit 112 determines instep S607 that the meal can be served to the assisted person B, thesupport information output unit 112 notifies the notification to thecaregivers to take a bite of the meal to the assisted person B in thestep S608. In this case, since the caregiver is performing the actionfor the assisted person B, the support information output unit 112performs processing to suspend the meal assistance sequence of theassisted person A until the action is completed.

Alternatively, the support information output unit 112 may suspend themeal assistance sequence related to the assisted person A until allother assisted persons in charge of the meal assistance by the samecaregiver are in the same position if it is determined that the assistedperson A is in the same position (Yes in the step S603).

FIG. 24A is a state transition diagram illustrating the transition ofthe assistance sequence for a prescribed caregiver. For example, thesupport information output unit 112 performs two meal assistancesequences to support the caregiver providing both the meal assistancefor the assisted person A and the assisted person B. In this case, thesupport information output unit 112 performs a state transition based ona prescribed condition. For example, the support information output unit112 stops the meal assistance sequence with respect to the assistedperson A and transitions to a state performing the meal assistancesequence with respect to the assisted person B if it is determined thatone assistance unit by the caregivers is completed when performing themeal assistance sequence for the assisted person A.

Alternatively, the support information output unit 112 may determine thepriority of the support information to be notified in each assistancesequence. For example, suppose that the support information output unit112 determines that a notification to record the meal result (the stepS610) is executed because the assisted person A had finished eating themeal and notification to serve a bite of the meal (the step S608) isexecuted since the assisted person A had not finished eating the meal.

The recording of the meal results can be done at any time until cleaningup, whereas the serving a bite of the meal should continue until theeating the meal for the assisted person B is completed. Therefore, inthis case, the support information output unit 112 may prioritize theexecution of the meal assistance sequence of the assisted person B andsuspend the meal assistance sequence of the assisted person A. In thisway, it is still possible to achieve appropriate state transitionsbetween multiple assistance sequences targeting multiple assistedpersons. It should be noted that the state transition between two ormore assistance sequences can be thought of as an interruption byanother assistance sequence to an ongoing assistance sequence.

In addition, although we have shown an example in which two mealassistance sequences are executed in parallel, the method of thisembodiment is not limited to this. FIG. 24B is another diagramillustrating the state transitions between the assistance sequences inthis embodiment.

As shown in the FIG. 24B, various sequences such as the meal assistancesequence, the excretion assistance sequence, the transferring or movingassistance sequence, and the abnormal response sequence may be executedin parallel in this embodiment. In this case, the support informationoutput unit 112 may control the transition between each assistancesequence shown in the FIG. 24B. In addition, the FIG. 24B shows anexample of a transition from a prescribed type of assistance sequence toanother type of assistance sequence via a standby state, but a directtransition may be executed between each assistance sequence. Also, asshown in the FIG. 24A, the meal assistance sequence may include multipleassistance sequences. Similarly, it is possible for other assistancesequences, such as the excretion assistance sequence to include multipleassistance sequences.

For example, suppose that a prescribed caregiver is serving a meal tothe assisted person A when the assisted person A is in an abnormalstate. The abnormal condition is, for example, choking. In this case,the caregiver will stop the meal assistance to the assisted person A andtake an action for the abnormal condition. For example, the supportinformation output unit 112 performs the start determination of theabnormality response sequence in the background in a same way of thesteps S501 to S503 in the FIG. 20 , and starts the abnormality responsesequence when the abnormality of the assisted person A is detected.Although the processing of the steps S505 to S506 in the FIG. 20 may beperformed, the processing of the steps S505 to S506 may be omitted inconsideration of the fact that the caregiver in charge of the assistedperson A is the same as the caregiver in charge of the meal assistanceand that there is a possibility of high emergency.

As the result, the abnormality response sequence may be added to theassistance sequence to be performed. Then, the support informationoutput unit 112 suspends the meal assistance sequence currently beingperformed and starts performing the abnormality response sequence. Whenthe abnormality is resolved by the abnormality response sequence, thesupport information output unit 112 performs a transition to anotherassistance sequence such as resuming the suspended meal assistancesequence.

Alternatively, in some cases, when a prescribed caregiver is serving themeal to the assisted person A, the assisted person A may want to go tothe restroom. In this case, the excretion assistance sequence is addedto the assistance sequence to be performed. In addition, depending onthe ADL of the assisted person A and the location of the toilet, atransferring or moving assistance sequence may be required. For example,the support information output unit 112 suspends the meal assistancesequence, firstly performs the transferring or moving assistancesequence to move to the toilet, then performs the excretion assistancesequence, and resumes the suspended meal assistance sequence that aftercompletion.

The factors that the necessary assistance changes include the initiativeof the assisted person, the physical condition of the assisted person,the diseases such as dementia, the medications, the environment, theseasons, the external factors, and the difference between the day's careprogress and the schedule. For example, the support information outputunit 112 may perform a processing to detect these factors and determinethe assistance sequence to be the transition destination based on thedetected factors and the assistance sequence currently being performed.

Thus, the support information output unit 112 can appropriately dealwith the various situations by performing multiple assistance sequencesin parallel and by controlling the state transitions between themultiple assistance sequences. For example, as noted above, even ifthere is a case that one caregiver deal with many assisted person, it ispossible to determine the assistance should be performed and the order.Since the burden on the caregiver can be reduced, the risk of incidentssuch as aspiration and falling of the assisted person can be reduced. Inaddition, even if other assistances suddenly become necessary duringperforming the prescribed assistance, it is possible to reduce theburden on the caregiver and the risk to the assisted person, because thecaregiver can appropriately support the assistance to be performed atthat time.

Adding Data by the Users

In the above explanation, it is assumed that the support informationoutput NN 122 is generated by the learning unit 114. For example, aprovider of an information processing device may select the nursinghome, etc. for learning in advance and may create the supportinformation output NN 122 using data acquired from the nursing home,etc. When a nursing home utilizing the services provided by theinformation processor is newly added, for example, the existing supportinformation output NN 122 is commonly used.

However, the method of this embodiment is not limited to this one, and anew training data may be added by the user of the nursing home, etc.,and the additional machine learning may be performed using the trainingdata.

For example, the data from each nursing home may be combined and usedfor machine learning while maintaining that the support informationoutput NN 122 is common among the multiple nursing homes. This case hasan advantage of increasing the number of training data because thetraining data can be collected from multiple nursing homes.

Alternatively, the additional machine learning may be performed for eachnursing home. In this case, the support information output NN 122 isupdated for each nursing home. That is, it becomes possible to make thesupport information output NN 122 specific to the target nursing home.

FIG. 25A is an example of a screen displayed on the display unit 214 ofa mobile terminal device 210, for example. In comparison with the FIG.17 , an object OB4 for adding the data is added. When the caregiverselects the object OB4, the screen changes to the screen of the FIG.25B.

The FIG. 25B includes an area RE1 showing the names of the supportinformation of the added training data, and an area RE2 which can inputthe assisted person ID, the caregiver ID and the output data. Theassisted person ID is the information to identify the assisted person.The caregiver ID is the information to identify the caregiver. Theoutput data is the information corresponding to the output of thesupport information output NN 122. Since the FIG. 25B deals with atiming of changing the diaper, an example of using time as output datais shown. However, the format of the output data can be modified invarious ways according to the type of the support information, and maybe an image, a voice, a number value, a binary data representingauthenticity, or other formats.

The example of the FIG. 25B shows that the time of 2021/MM/DD hh: mm: ssas the timing of changing the diaper was appropriate when the caregiverwhose caregiver ID is “abcde” performs the excretion assistance for theassisted person whose assisted person ID is “12345”. Apart from thecaregiver's operation, the input data corresponding to the timing ofchanging the diaper is acquired in the nursing home. That is, the dataset in which the input data is associated with the output data of2021/MM/DD hh: m m: ss may be a training data of the support informationoutput NN 122 which outputs a timing of changing the diaper.

However, the present embodiment assumes that the tacit knowledge of theskilled assistant is converted into data and that the appropriateassistance is performed regardless of the skill level of the assistant.Therefore, even if the above data set is obtained by the input from aprescribed caregiver, it is not clear whether it is a positive data or anegative data. The positive data represents a data set with appropriatecorrect data associated with the input data, and the negative datarepresents a data set with inappropriate correct data associated withthe input data.

Therefore, the learning unit 114 may store an association information inwhich the caregiver ID for example, is associated with the skill levelof the caregiver. The level of skill may be manually input by a nursinghome administrator, or it may be automatically determined based on yearsof experience, the qualifications, and the past nursing history. Thelearning unit 114 sets the data set by the highly skilled caregiver asthe positive data and the data set by the less skilled caregiver as thenegative data.

Alternatively, even when the assistance is provided by a skilled person,the assistance can be considered as either the case that the assistanceis provided according to the defined procedure or the case theassistance is adjusted by one's own instinct. It is highly probable thatthe tacit knowledge of a skilled person is used when the skilled persontakes the actions according to a hunch. Therefore, as shown in the FIG.25B, the area RE2 of the display screen may be input-able whether or nota hunch was used. The caregiver inputs into the area RE2 whether or nothe or she had used a hunch, for example, when determining the timing ofchanging the diaper. The learning unit 114 uses the data set when thecorresponding input is “yes” as the positive data.

Regarding to the learning process after acquiring the training data, adetailed explanation is omitted because it is the same as the exampledescribed above using the FIG. 8 .

In the example of the FIG. 25B, by updating the support informationoutput NN 122 that outputs the timing of changing the diaper, it becomespossible to output more accurate the timing of changing the diaper. Now,we have explained an example of the timing of changing the diaper, butadding training data is equally possible for other support information.

Custom Support Information

In the above, the FIGS. 43 to 45 are illustrated as the supportinformation that can be output. However, as we can see from the aboveexplanation, there is a wide variety of the support required for theassistance, and it is possible that the support required for theassistance may vary depending on the nursing home or the caregiver. As aresult, there may be cases in which the support information of a typenot included in the existing support information is needed. Therefore,in this embodiment, the caregiver may be able to add any custom supportinformation.

For example, in the FIG. 25B, the name of the support informationdisplayed in the area RE1 is not fixed and may be freely editable by thecaregiver. The caregiver inputs the name of the desired custom supportinformation using text such as “a timing of performing xxxx.” “xxxx” isa text that describes a detailed assistance action performed by, forexample, a caregiver. In addition, when the caregiver performs theassistance action corresponding to “xxxx,” the caregiver ID, theassisted person ID, the output data, whether or not the hunch was used,etc., are input. Thus, as part of the training data of the supportinformation output NN 122 that outputs “a timing of performing xxxx,”the output data and the information indicating whether the output datais the positive data or the negative data are obtained.

In addition, the information processing device may control to display ascreen to identify the input data in the training data on the displayunit 214 of the mobile terminal device 210. The FIG. 25C is an exampleof a display screen for specifying the input data. The screen shown inthe FIG. 25C includes an area RE3 for displaying the name of the customsupport information and an area RE4 where the name of the device alreadyplaced in the target nursing home or the name of the input data acquiredby the device can be selected.

For example, a sleep scan is a sensing device 450 shown in the FIG. 2D,which can detect a heart rate, a respiratory rate and an activity. Thecaregiver selects the data available on the device that he or she wantsto use as input data when obtaining the custom support information amongthe data which can detect by the device. The FIG. 25C shows an examplethat the caregiver selects to take the breathing rate from the sleepscan, to take the image arranged on the bedside camera, not to take theoutput of the pulse oximeter as the input data.

By using the screen shown in the FIG. 25C, the name of the customsupport information is associated with the input data used to output thecustom support information. The association information representingthis association is transmitted to the server system 100 and stored inthe storage unit 120.

The storage unit 120 of the server system 100 stores the time series ofbreathing rates data and the time series of the images of the bedsidecamera. Therefore, the learning unit 114 extracts the respiratory rateand the camera image corresponding to the output data acquired using theFIG. 25B as input data.

For example, the server system 100 keeps the timing of acquiring theoutput data based on the FIG. 25B, and reads the respiration rate andthe camera image for a prescribed period set based on the timing fromthe storage unit 120. Then, the learning unit 114 performs a learningprocessing of the support information output NN 122 to output the customsupport information based on the training data in which the read inputdata is associated with the output data.

Also, as shown in the FIG. 25C, the object OB5 for starting the learningoperation may be displayed on the display unit 214 of the mobileterminal device 210. When it is detected that the caregiver has selectedthe object OB5, the learning unit 114 performs the above learningprocessing. Thus, a new support information output NN 122 to output thecustom support information is generated. Since the learning process isthe same as the above example, a detailed explanation is omitted. Inaddition, in the machine learning of the custom support information, itmay be sufficient that the training data in which the input data isassociated with the output data can be obtained, and the user interfaceis not limited to those described above.

The structure of the NN can be modified in various ways. The FIG. 26shows the structure of the general NN. The NN shown in the FIG. 26includes a CNN1 which extracts feature quantities using the image dataas input, a CNN2 which extracts feature quantities using the voice dataas input, a vector transformation NN which extracts feature quantitiesusing text data as input, and CNN3 which extracts feature quantitiesusing other sensor information as input. The NN in the FIG. 26 alsoincludes a DNN (Deep Neural Network) that accepts the outputs from theCNN1, the CNN2, the vector transform NN, and the CNN3 and outputs thecustom support information.

The NN shown in the FIG. 26 can accept the image, the voice, the textand other sensor information as the inputs. The input data of the customsupport information can have various patterns as shown in, for example,FIG. 25C, but the NN shown in the FIG. 26 can appropriately accept dataof any pattern as the input data. If the image data is not selected asinput data, the input of CNN1 is treated as 0. The same applies when thevoice data, the text data or other sensor information is not selected asinput data, and the input of the corresponding NN among the CNN2, thevector conversion NN and the CNN3 becomes 0.

FIG. 25D shows an example of a screen displayed on the display unit 214of the mobile terminal device 210 after the completion of the machinelearning. The display screen of the FIG. 25D displays the correct answerrate obtained using the validation data, for example, in the learningprocess. In the example shown in the FIG. 25D, the caregivers can choosewhether or not to output the custom support information using thislearning result. For example, if the caregiver selects “yes” to thequestion “Do you want to apply?”, the custom support information can beoutput. For example, similar to the example described above in the FIG.17 , the custom support information can be output by setting “active”about the custom support information “a timing of performing xxxx.” Onthe other hand, if the caregiver selects “no”, no custom supportinformation will be output.

It is also possible that caregivers want to use the target customsupport information because it is important, although the target customsupport information can not be adopted as is due to the low accuracyrate. In this case, it may be possible to request the analysisprocessing to the manager or the provider of the information processingdevice. For example, when the caregiver select “yes” for the question,“Would you like to request an analysis?”, a changing processing of thesupport information output NN 122 to output the custom supportinformation is performed on the server system 100 side.

The learning unit 114 of the server system 100 may try to improve thecorrect answer rate by changing the structure of the NN, for example,when the original correct answer rate is lower than a prescribedthreshold. This is because the NN shown in the FIG. 26 has aconfiguration that considers the versatility as described above, and theaccuracy rate may be improved by having a structure that is morespecialized for the custom support information. If the original correctanswer rate exceeds the prescribed threshold, the learning unit 114 mayskip the change processing of the support information output NN 122.

For example, when there are multiple NNs with different structures fromeach other as the support information output NN 122 for as shown in theFIG. 10 , the learning unit 114 may classify the multiple NNs intoseveral classes.

FIG. 27 is a diagram to explain the classification process of the NN.For example, the learning unit 114 obtains an n-dimensional featurequantity by performing a text mining processing using text representingthe name of the support information as the output, and performs aclustering processing based on the n-dimensional feature quantity. Forconvenience of the explanation, a two-dimensional feature plane is shownin the FIG. 27 , but n may be 3 or more. For example, among a pluralityof the support information output NN 122, if an NN outputting “a timingof changing the diaper” is targeted, words such as “diaper”, “change”and “timing” are extracted, and an n-dimensional feature quantity of theNN outputting “a timing of changing the diaper” is obtained based on theextraction result.

The clustering processing method is not limited to the text miningprocessing, and the learning unit 114 may cluster multiple NNs byperforming an analytical processing such as logistic regressionanalysis. The learning unit 114 may also assign the clustering resultswhich are manually clustered for the part of the multiple NNs shown inthe FIG. 10 and perform the clustering processing for the remaining NNsusing the clustering results. In this way, the accuracy of theclustering processing can be improved.

In the example in the FIG. 27 , among the multiple NNs stored by theserver system 100, the NN1 to NN3 were classified as the class 1, theNN4 to NN7 were classified as the class 2, and the NN8 to NN10 wereclassified as the class 3. The learning unit 114 determines which classit belongs to by similarly obtaining the n-dimensional feature quantitybased on the name of the custom support information. For example, thelearning unit 114 extracts words such as “XXXX” and “timing” from thename of custom support information such as “a timing to performingXXXX,” and obtains an n-dimensional feature quantity corresponding tothe custom support information based on the extraction result. Thelearning unit 114 determines the structure of the NN used for thelearning based on the clustering result of the custom supportinformation.

For example, as shown in the FIG. 27 , it is assumed that the customsupport information is classified as the class 1. In this case, thelearning unit 114 selects any one of the NN1 to NN 3 and generates a NNfor the custom support information using the structure of the selectedNN and the training data for the custom support information describedabove. Only the structure of the original NN is used, and all of theweights may be computed anew. Alternatively, a transfer learning may beperformed using a part of the original NN's weights as is. For example,the learning unit 114 performs a machine learning using the respectivestructures of the NN1 to NN 3 and training data for the custom supportinformation, and obtains the correct answer rate of the learned model.Then, the learning unit 114 presents the highest correct answer rate tothe caregiver in the same manner as in the FIG. 25D and makes thecaregiver input whether or not the learned model is applied. When thecaregiver responds “yes”, the corresponding support information outputNN 122 is stored in the storage unit 120 to enable the output of thecustom support information.

When an additional machine learning is performed, it is important thatthe relationship between the period of the accumulation of the trainingdata, in other words, the period of the acquired data to be analyzed,and the ADL of the assisted person. For example, suppose that anassisted person who was able to take actions independently broke a bonein the falling and he or she need the assistance by a wheelchair. Ifthere is a significant change in ADL, there is a significant differencein the appropriate assistance for the assisted person between before thechange and after the change. Therefore, for example, the learningresults used the training data before changing ADL may not be usefulafter changing ADL.

Therefore, although not shown in the FIG. 25C, for example, whenstarting performing a learning processing, it may be possible to inputnot only the type of the input data but also the analysis period. Thecaregiver designates a period of time during which the assisted person'sADL is considered to be comparable to the current one. In this way, thesupport information output NN 122, which is the result of the learningprocessing, can support the appropriate assistance because the contentof the learned model corresponds the current ADL of the assisted person.It is also assumed that the server system 100 collects the ADL of theassisted person, for example, as one of the input data. Therefore, whenstarting performing the learning processing, the learning unit 114 mayacquire the time series change of the ADL of the target assisted personand automatically set the analysis period based on the time serieschange of the ADL.

Although the present embodiment has been described in detail asdescribed above, those skilled in the art will readily understand thatmany modifications can be made that do not materially deviate from thenovel matters and effects of the present embodiment. Therefore, all suchvariations shall be included in the scope of this disclosure. Forexample, a term appearing at least once in a description or drawing witha different term that is more broadly or synonymously may be replaced bythat different term anywhere in the description or drawing. Allcombinations of this embodiment and variations are also included in thescope of this disclosure. Moreover, the configuration and operation ofthe information processing system, the server system, the mobileterminal device, etc., are not limited to those described in thisembodiment, and various modifications can be performed.

1. An information processing device comprising: a factor determinationunit configured to determine whether a behavior of an assisted person isan abnormal behavior of a dementia factor based on (1) an information ona dementia level of the assisted person and (2) at least one of anenvironmental information, an excretion information, and a sleepinformation of the assisted person, and a support information outputunit configured to output a support information to support an assistanceof the assisted person by a caregiver based on the determination resultof the factor determination unit and a sensor information that is asensing result about the assisted person or the caregiver assisting theassisted person.
 2. The information processing device according to theclaim 1, wherein the support information includes an information tosupport at least one of a meal assistance to assist the assisted personto eat a meal, an excretion assistance to assist the assisted person toexcrete, and a transferring or moving assistance to assist the assistedperson to transfer or move.
 3. The information processing deviceaccording to the claim 2, wherein the support information output unit isconfigured to output an information indicating the amount of the meal tobe served per one bite and an information indicating a timing of servingthe meal per one bite in the meal assistance as the support information,and is configured to change at least one of the amount of serving andthe timing of serving in comparison with a case the behavior isdetermined not to be the abnormal behavior of the dementia factor if thebehavior is determined to be the abnormal behavior of the dementiafactor.
 4. The information processing device according to the claim 2,wherein the support information output unit is configured to output aninformation specifying a timing to start performing the excretionassistance as the support information, and is configured to change thetiming to start performing the excretion assistance in comparison with acase the behavior is determined not to be the abnormal behavior of thedementia factor if the behavior is determined to be the abnormalbehavior of the dementia factor.
 5. The information processing deviceaccording to claim 1, wherein the support information output unit isconfigured to increase a type of the sensor information in comparisonwith a case the behavior is determined not to be the abnormal behaviorof the dementia factor if the behavior is determined to be the abnormalbehavior of the dementia factor.
 6. The information processing deviceaccording to the claim 5, wherein the support information output unit isconfigured to determine whether or not to add a new sensor based on aninformation specifying one or more used sensors and a sensor informationto be added when the behavior is determined to be the abnormal behaviorof the dementia factor.
 7. An information processing method comprising:a factor determining step to determine whether a behavior of an assistedperson is an abnormal behavior of a dementia factor based on (1) aninformation on a dementia level of the assisted person and (2) at leastone of an environmental information, an excretion information, and asleep information of the assisted person, and an outputting step tooutput a support information to support an assistance of the assistedperson by a caregiver based on the determination result of the factordetermining step and a sensor information that is a sensing result aboutthe assisted person or the caregiver assisting the assisted person. 8.The information processing device according to claim 2, wherein thesupport information output unit is configured to increase a type of thesensor information in comparison with a case the behavior is determinednot to be the abnormal behavior of the dementia factor if the behavioris determined to be the abnormal behavior of the dementia factor.
 9. Theinformation processing device according to claim 3, wherein the supportinformation output unit is configured to increase a type of the sensorinformation in comparison with a case the behavior is determined not tobe the abnormal behavior of the dementia factor if the behavior isdetermined to be the abnormal behavior of the dementia factor.
 10. Theinformation processing device according to claim 4, wherein the supportinformation output unit is configured to increase a type of the sensorinformation in comparison with a case the behavior is determined not tobe the abnormal behavior of the dementia factor if the behavior isdetermined to be the abnormal behavior of the dementia factor.